Asyncifying django.contrib.auth? (and signals, and maybe sessions)

Hey everyone, I’m working on a plan to try and asyncify django.contrib.auth. I’ve chatted about this on the django-developers mailing list ( and it’s starting to crystalize into a plan so Carlton thought it might be a good idea to move discussion over here.

The basic need I have is that my entire django installation is async. I do this because its simpler for me to reason about one or the other instead of both, and also because I’m a heavy GraphQL user so concurrent loading is much more efficient. (see the email thread for more about my personal stake in this). One of 2 remaining non-async callsites from my code into Django is interaction with the auth app (the other is raw database connections, which are a long way from async, and not relevant here). I’d like to clean this up to be async-native, then make it efficient in async (minimize sync/async context switches).

The basic thesis of the plan is to asyncify the entire auth app so that when it is running under ASGI there are as few sync/async context switches as possible while keeping the existing synchronous interface intact.

I’ve prepared a not-meant-for-review PoC of this here: Dummy implementation of async interface for `django.contrib.auth` by bigfootjon · Pull Request #1 · bigfootjon/django · GitHub (note this is a PR from my fork to the fork)

That PR is just to illustrate what I mean when I say the “entire” app: the API, the middleware, and the backends. The current version does not work because of some needs to asyncify “upstream” components like the sessions contrib app and signals. But I’d like to think that the dummy PR I made there is a sketch of the final version of this plan.


Here is my proposal in detail:

  1. Add an async interface to the auth app’s API

This would require creating async versions of the following functions authenticate, login, logout, get_user, update_session_auth_hash. This matches a lot of other ongoing work in QuerySet and related ORM classes to add async wrappers around sync methods with sync_to_async doing most of the work and doesn’t seem controversial.

  1. Asyncify signals (Ticket 32172)

This doesn’t immediately seem related, but it would be a prerequisite to asyncifying the internals of the auth app. The implementation provided in the PR linked to the above ticket is almost workable, and I think with a little elbow grease we can get it into a mergeable state.

  1. Asyncify sessions? (no ticket, afaict)

This is just a repeat of signals, though this is a bit more difficult due to a larger API surface with more decisions to be made, so I put it after signals (also, might be worth separate discussion on its own to see if this is worthwhile).

  1. Asyncify auth backends/internals

Once the prereqs are done (2 & 3), we could asyncify the internals of the auth app. This would allow for async-compatible backends and then take advantage of them to make the internals of the auth app fully async (if called from an async API entrypoint, like the alogin function that would be implemented in (1)).

Note that I said async-COMPATIBLE. I envision that subclasses of auth.backends.BaseBackend would have several new functions added to support async-mode while still supporting sync-mode in the same classes. Default implementations of async-mode would be added automatically via sync_to_async, and implementers are free to choose to add async support or leave the default, slightly inefficient implementation in place.

The “backend interface and impl” commit in my dummy PR illustrates what I’m talking about.

  1. Resolve ticket 31920

This ticket blocks full asyncification of the middleware, so I thought it was worth calling out as a pre-req to (6).

  1. Asyncify auth middleware

Once (5) is resolved, we can update the implementations of AuthenticationMiddleware and RemoteUserMiddleware to be “async native” in ASGI mode and not need a sync/async context switch.

Open Questions

After (6), auth should be async from top to bottom, while preserving the existing sync interface. I think there are a few open questions that Carlton and I discussed and are worth restating here:

A. Ticket 31949

This ticket is about adding async-compatible decorators, and touches on the auth decorators. Carlton and I have concluded this is an orthogonal issue to the one addressed in this topic, but I wanted to re-raise it in case anyone has any thoughts here.

B. Are there proven strategies for reducing code duplication between sync and async versions of functionality in Django or in Python broadly? We’re not aware of any guidance here but I’m eager for resources to consider. The dummy implementation I proved above is verbose (doubles file size in some cases) and fragile (what if a bug fix is only applied to the sync version and not the async version?) right now.

C. Is there anything I’m missing? As Carlton said “No plan ever survives first contact with the enemy :)” but there might be more obvious pitfalls that we can try and dodge before getting started.

Any other questions or comments? My entire contribution history with Django up to now has been commenting on a few tickets and contributing 2 fairly trivial PRs. I apologize if I’m making any etiquette or protocol mistakes, but I’m eager to help out.

I’d be happy to take on (1) if it isn’t controversial, and (2) seems like something outside my comfort zone that is attainable, and (3) might feel less daunting after (2). After (3) its a bit too hazy to see how things would fall, but I’d like to think I could take those on too with a little help.

I’m mostly looking for guidance, counter arguments, or hot/cold takes. If anyone is interested in collaborating on this work I’d welcome that too!


1 Like

Just to keep this thread up to date, I’ve gone ahead and submitted some PRs to handle a few bits from the above plan:

  1. Fixed #32172 -- Adapted signals to allow async handlers. by bigfootjon · Pull Request #16547 · django/django · GitHub (#2 in the plan)
  2. Fixed #31920 -- Updated AuthenticationMiddleware to add request.auser by bigfootjon · Pull Request #16552 · django/django · GitHub (#5 in the plan)

I’m aware I’ve sort of jumped the gun here, but I only have opportunities to work on Django in spurts and this seemed like things that were already under way and could be done independent of whether or not this overall plan is taken on.

Ah shoot, I just realized I posted this in the wrong part of the forums. This should be in the “internals/async” category. Is it possible to move this topic over there?

Your friendly neighborhood moderators can take care of that. (Yea, it’s done, but replies need to be 20 characters, so I blab a bit.)

1 Like

Both of the above PRs that I mentioned have been merged.

I’ve gone ahead and filed a new ticket (#34391 (Add async interface to contrib.auth) – Django) to track work on an async auth interface, and put a PR (Fixes #34391: Add async interface to auth by bigfootjon · Pull Request #16636 · django/django · GitHub) up for review with the implementation.

That means that in my above proposal:
#1: In progress, PR is waiting for review
#2: Done
#3: Pending creation of a ticket, I’d like to understand the scope of work before I file that
#4: Blocked on #3, and might need more research into reducing code duplication before starting
#5: Done
#6: Partially done during #5, just needs some tweaks to RemoteUserBackend after #1 is done

#1 and #3 can be worked on right now, I’m focused on #1 for now. #3 is available if anyone stumbles on to this thread and is interested in contributing!