RFC: Stop using request.META syntax to configure header names

Django Internals
1

Many of Django’s settings predate request.headers, which provides a less-WSGI-specific interface to HTTP headers. The settings instead use keys of request.META (which comes from the WSGI environ). Even those added after request.headers use the request.META syntax by convention.

This rarely causes problems, but can cause confusion for new developers who have to provide a different spelling for headers to what they may be used to.

Additionally, the WSGI environ can be modified by the WSGI server directly, which isn’t true under ASGI. #36862 (Clarify RemoteUserMiddleware usage and deployment requirements under ASGI) – Django highlights that in its default configuration, RemoteUserMiddleware reads request.META["REMOTE_USER"] under WSGI, but request.META["HTTP_REMOTE_USER"] under ASGI. The former is safe, since it doesn’t correspond to a header, however the latter is “less safe” (I’m intentionally not calling it “unsafe”), since it does correspond to a header, and could lead to a vulnerability if the fronting proxy if the header isn’t properly sanitized.

Therefore, I propose changing Django’s settings to instead specify headers based on the verbatim header name (eg X-Bender) and reading from request.headers rather than request.META key (HTTP_X_BENDER). It avoids confusing when setting and keeps a consistent implementation between ASGI and WSGI (and future SGI’s). request.headers is derived from request.META but with appropriate normalisation, and is derived strictly from HTTP headers.

For ease and convenience, this change should be applied both to values from settings, but also any internal header references (perhaps in stages).

Solutionizing, to avoid this being a huge breaking change, settings would need to support both spellings:

_is_meta_header = re.compile(r"HTTP_[A-Z1-9_]+")

if _is_meta_header.match(settings.CSRF_HEADER_NAME):
    # Raise deprecation warning
    value = request.META[settings.CSRF_HEADER_NAME]
else:
    value = request.headers[settings.CSRF_HEADER_NAME]

(Don’t read too much into the implementation. A regex may not be the best way, and the match should definitely be pre-computed. This is just an example to show it should follow a deprecation path).

1 Like
2

Hey @theorangeone

I (half?) wish I hadn’t happened to respond to a ping in another thread and then seen this :sweat_smile:

I understand the desire.

From experience, I predict this will be one of those changes that has much wider fallout and annoyance than I imagine you to be thinking about. tl;dr Folks will not see or ignore the warning and then we’ll get numerous reports about breakages when the fallback is removed.

People will be cross. It’s of marginal benefit: they might be right to be. (c.f. “I understand the desire”)

As such my immediate gut reaction is in the -1 to -0 range.

If the Make django-upgrade official story were further along/already proven, I’d (assuming it proves a success) be more inclined. (I didn’t think if/how it would help in this exact case.)

Maybe a purely opt-in as_wsgi_format(header) helper could be useful. Such could just become a no-op if/when we solve the deprecation issue. (Not sure I’m convinced, but it came to mind)

:person_shrugging:

3

The papercut, and my motivation for supporting it, is that it doesn’t support the header name, which is the obvious thing to put in as a value. Matching intuition makes things easier for learners to use Django. The caveats in this case (headers that are bizarrely named starting with HTTP_ ) don’t strike me as a big concession if we want to prioritize both stability as well as removing this papercut.

4

django-upgrade cannot 100% upgrade settings patterns, as it’s all heuristics. I think it would be okay here but not great.

I renamed some of the settings over in django-cors-headers, and will leave in the “read the old name” code paths forever, e.g. CORS_ALLOW_ALL_ORIGINS was called CORS_ORIGIN_ALLOW_ALL. But I’m not sure that’s the best approach.

@theorangeone your post doesn’t concretely name the affected settings. From searching for HTTP_ in the docs, I think they’re CSRF_HEADER_NAME and SECURE_PROXY_SSL_HEADER. That’s only two “pro-level” settings that would be affected: some warts seem permissible then.

So I think I’m also in the -1 to -0 range…

1 Like