Hey everyone, I’m working on a plan to try and asyncify django.contrib.auth. I’ve chatted about this on the django-developers mailing list (https://groups.google.com/g/django-developers/c/T8zBnYO78YQ) and it’s starting to crystalize into a plan so Carlton thought it might be a good idea to move discussion over here.
The basic need I have is that my entire django installation is async. I do this because its simpler for me to reason about one or the other instead of both, and also because I’m a heavy GraphQL user so concurrent loading is much more efficient. (see the email thread for more about my personal stake in this). One of 2 remaining non-async callsites from my code into Django is interaction with the auth app (the other is raw database connections, which are a long way from async, and not relevant here). I’d like to clean this up to be async-native, then make it efficient in async (minimize sync/async context switches).
The basic thesis of the plan is to asyncify the entire auth app so that when it is running under ASGI there are as few sync/async context switches as possible while keeping the existing synchronous interface intact.
I’ve prepared a not-meant-for-review PoC of this here: Dummy implementation of async interface for `django.contrib.auth` by bigfootjon · Pull Request #1 · bigfootjon/django · GitHub (note this is a PR from my fork to the fork)
That PR is just to illustrate what I mean when I say the “entire” app: the API, the middleware, and the backends. The current version does not work because of some needs to asyncify “upstream” components like the sessions contrib app and signals. But I’d like to think that the dummy PR I made there is a sketch of the final version of this plan.
Proposal
Here is my proposal in detail:
- Add an async interface to the auth app’s API
This would require creating async versions of the following functions authenticate, login, logout, get_user, update_session_auth_hash. This matches a lot of other ongoing work in QuerySet and related ORM classes to add async wrappers around sync methods with sync_to_async doing most of the work and doesn’t seem controversial.
- Asyncify signals (Ticket 32172)
This doesn’t immediately seem related, but it would be a prerequisite to asyncifying the internals of the auth app. The implementation provided in the PR linked to the above ticket is almost workable, and I think with a little elbow grease we can get it into a mergeable state.
- Asyncify sessions? (no ticket, afaict)
This is just a repeat of signals, though this is a bit more difficult due to a larger API surface with more decisions to be made, so I put it after signals (also, might be worth separate discussion on its own to see if this is worthwhile).
- Asyncify auth backends/internals
Once the prereqs are done (2 & 3), we could asyncify the internals of the auth app. This would allow for async-compatible backends and then take advantage of them to make the internals of the auth app fully async (if called from an async API entrypoint, like the alogin function that would be implemented in (1)).
Note that I said async-COMPATIBLE. I envision that subclasses of auth.backends.BaseBackend would have several new functions added to support async-mode while still supporting sync-mode in the same classes. Default implementations of async-mode would be added automatically via sync_to_async, and implementers are free to choose to add async support or leave the default, slightly inefficient implementation in place.
The “backend interface and impl” commit in my dummy PR illustrates what I’m talking about.
- Resolve ticket 31920
This ticket blocks full asyncification of the middleware, so I thought it was worth calling out as a pre-req to (6).
- Asyncify auth middleware
Once (5) is resolved, we can update the implementations of AuthenticationMiddleware and RemoteUserMiddleware to be “async native” in ASGI mode and not need a sync/async context switch.
Open Questions
After (6), auth should be async from top to bottom, while preserving the existing sync interface. I think there are a few open questions that Carlton and I discussed and are worth restating here:
A. Ticket 31949
This ticket is about adding async-compatible decorators, and touches on the auth decorators. Carlton and I have concluded this is an orthogonal issue to the one addressed in this topic, but I wanted to re-raise it in case anyone has any thoughts here.
B. Are there proven strategies for reducing code duplication between sync and async versions of functionality in Django or in Python broadly? We’re not aware of any guidance here but I’m eager for resources to consider. The dummy implementation I proved above is verbose (doubles file size in some cases) and fragile (what if a bug fix is only applied to the sync version and not the async version?) right now.
C. Is there anything I’m missing? As Carlton said “No plan ever survives first contact with the enemy :)” but there might be more obvious pitfalls that we can try and dodge before getting started.
Any other questions or comments? My entire contribution history with Django up to now has been commenting on a few tickets and contributing 2 fairly trivial PRs. I apologize if I’m making any etiquette or protocol mistakes, but I’m eager to help out.
I’d be happy to take on (1) if it isn’t controversial, and (2) seems like something outside my comfort zone that is attainable, and (3) might feel less daunting after (2). After (3) its a bit too hazy to see how things would fall, but I’d like to think I could take those on too with a little help.
I’m mostly looking for guidance, counter arguments, or hot/cold takes. If anyone is interested in collaborating on this work I’d welcome that too!
Thanks!