process_exception is not hit when CsrfViewMiddleware middleware is used

Using Django Forms & APIs
1

Hi!

I am quite new to python and django.

I am trying to figure out why in our project I can’t handle exceptions via the middleware’s process_exception method if CsrfViewMiddleware is used.

I can run into my custom middleware’s process_exception hook until CsrfViewMiddleware is not added to the project (but it is a necessity).

Can somebody explain why I experience this?

2

I think we might need to see more details regarding the behavior you’re experiencing - including (possibly) your middleware’s process_exception hook, and the exception that you are catching and handling.

It may be something as simple as a “sequence of middleware” issue, where CsrfViewMiddleware is handling something you’re expecting to see - or vice-versa.

But it’s going to be impossible to tell without knowing more of the context around this situation.

3

Thank you for the quick reply! I want to catch not RequestDataTooBig error!

in settings.py:

MIDDLEWARE = [
  'csp.middleware.CSPMiddleware',
  'django.middleware.security.SecurityMiddleware',
  'django.middleware.clickjacking.XFrameOptionsMiddleware',
  'django.middleware.common.CommonMiddleware',
  'django.contrib.sessions.middleware.SessionMiddleware',
# This CsrfViewMiddleware causes the problem, 
#if it is included anywhere in the list (top/bottom etc), 
# when I call my endpoint with a known error scenario,
# the error won't be processed.
# If I comment it out, everything works. 
# Nothing complicated, no twist, just the presence of this middleware!
  'django.middleware.csrf.CsrfViewMiddleware',
  'django.contrib.auth.middleware.AuthenticationMiddleware',
  'django.contrib.messages.middleware.MessageMiddleware',
  'CUSTOM.middleware.ExceptionHandler',
]

Our custom Exception Handler class is:

I want to catch not RequestDataTooBig error!

middleware.py

class ExceptionHandler(object):
  def __init__(self, get_response):
    self.get_response = get_response

  def __call__(self, request):
    self.process_response(request)
    response = self.get_response(request)
    return response
    
  def process_exception(self, request, e):
    if isinstance(e, PermissionException):
      return HttpResponse(str(e), status=500)
    elif isinstance(e, SuspiciousFileOperation):
      LOGGER.exception(e)
      return HttpResponse('invalid path')
    elif isinstance(e, PermissionDenied):
      return HttpResponse('Forbidden', status=403)
    LOGGER.exception("Exception on request processing")
    if isinstance(e, RequestDataTooBig):
      LOGGER.debug("==== RequestDataTooBig todo: raise error with the limit ======== ")
      raise e
    if getattr(settings, 'DEBUG', False):
      raise e
    if isinstance(e, Http404):
      raise e
    return HttpResponse(str(e), status=500)
4

I haven’t traced my way all the way through the CsrfViewMiddleware, but it seems reasonable to me to guess that it’s encountering that error condition and blocking the request from going further down the stack.

It looks to me that in the case of any error, that middleware calls the view identified by the setting CSRF_FAILURE_VIEW, which would bypass all other middleware processing.

You may need to put your exception handler above this to catch that error yourself. Or, if all you’re doing is logging this error, you could change your logger to make sure it’s being logged. (see SuspiciousOperation)

5

Thank you!

I digged into it, and pyenv/versions/v3.9.0/lib/python3.9/site-packages/django/core/handlers/exception.py: convert_exception_to_response swallows it:

def convert_exception_to_response(get_response):

Wrap the given get_response callable in exception-to-response conversion.

All exceptions will be converted. All known 4xx exceptions (Http404,
PermissionDenied, MultiPartParserError, SuspiciousOperation) will be
converted to the appropriate response, and all other exceptions will be
converted to 500 responses.

This decorator is automatically applied to all middleware to ensure that
no middleware leaks an exception and that the next middleware in the stack
can rely on getting a response instead of an exception.

From this, the only thing seems a question: why only that middleware is swallowing it:) I expected all DJANGO default middleware to run into this phase.

The way I handled it to use a custom middleware which force this error to be raised and return before any other middleware can swallow it. My only concern that is not happening true the real process_exception, but fine with me.
BTW not my idea:

(not my idea: python - Django: catch RequestDataTooBig exception - Stack Overflow

6

Because the csrf middleware doesn’t return the exception.

It wraps its own process_view within a try/except block, catching its own defined RejectRequest exception, returning self._reject(...) - which ends up calling the CSRF_FAILURE_VIEW.

So if the view defined by CSRF_FAILURE_VIEW doesn’t return an exception, the exception won’t be seen by the other middleware.