That’s awesome! @HappyDingning have you been working on prototyping a version of this? If not, maybe we could build off of @Andrew-Chen-Wang’s work as a starting point
A few thoughts/clarifying pointers after reading through this thread:
- This was on a FastAPI backend, so context switching has not been tested. My initial thoughts on the session store implementation above is that, under a single class, other modules don’t need to access two different session store objects. With two classes, there would be potential for race conditions with the
savemethod unless you shared a storage method such as contextvars. General thoughts on the implementation is that it’s safe. - The methods don’t try to touch any Python magic methods which is the typical case for most asyncio implementations I’ve added over the past year (especially
__getitem__,__del__, etc.). I warn against those because you never know when an async only call is needed that can mess those up. Best be explicit. In the redis-py implementation, this is also discouraged and simply not allowed (including in the sans-io rewrite). - Separate classes topic:
tl;dr use a single class in this case per point 1
I’m open to the separate class idea too, code wins arguments.
Generally, this should definitely be allowed in certain contexts. Even in Django, the test Client class has an AsyncClient counterpart, or ASGIRequest vs. WSGIRequest. Additionally, in the db driver, my (albeit long-idle draft) PR has a separate async class from its sync counterparts (even if it uses the same package like psycopg’s compatibility for both sync+async).
The case to use two separate classes for sync/async is not obvious in Django given precedence. In my opinion, when you’re stuck in a certain context (like async testing only with AsyncClient), it’s best to have two different classes for sync and async to optimize for the specific environment; furthermore, if a local storage is unneeded, separate classes can provide efficiency while still following DRY. But there are a lot of nuances (such as the implementation of LocMem cache + Redis cache backends, but I digress for a different thread).
Good spot to see the context-separate-class philosophy is in redis-py’s or elasticsearch-py’s asyncio module, but they’re following DRY principle + bound by low-level I/O connections.
Feel free to ping/email me with questions or if more code is needed; unlikely to have time to visit implementation.
Andrew
1 Like
I’ve opened #34901 (Add async interface to contrib.sessions) – Django which will track work to merge @Andrew-Chen-Wang 's prototype into main (which would be released in 5.1).
Just as a summary of where this lands in the work I outlined in the original post:
- DONE
- DONE
- The just-filed ticket (#34901 (Add async interface to contrib.sessions) – Django)
- Blocked by (3)
- DONE
- Blocked by (4)
2 Likes
I traced a problem we encountered with upgrading Django to v5.0 in buildroot to Refs #31949 -- Made @sensitive_variables/sensitive_post_parameters decorators to work with async functions. by bigfootjon · Pull Request #16831 · django/django · GitHub.
Maybe someone here has an idea how to resolve this?
CC @bigfootjon who wrote the PR above. Thanks for your help! 
Just to close the loop here, #35187 (@sensitive_variables/sensitive_post_parameters decorators crash with .pyc-only builds.) – Django has been solved via Fixed #35187 -- Fixed @sensitive_variables/sensitive_post_parameters decorators crash with .pyc-only builds. by felixxm · Pull Request #17860 · django/django · GitHub and has been cherry-picked back to the 5.0.x branch for release with the next bugfix release of Django.
2 Likes
A few updates on this thread:
Django now has a pyc-only daily test: Refs #30950, Refs #35187 -- Added tests for byte-compiled Django to daily builds. by bigfootjon · Pull Request #17870 · django/django · GitHub. This will hopefully prevent issues like #35187 from cropping up in the future.
Fixed #34901 -- Added async-compatible interface to session engines. by bigfootjon · Pull Request #17372 · django/django · GitHub has been merged which closes out #34901 (aka number 3 above)
I’ve just filed a new ticket for step (4): #35303 (Add async implementations to contrib.auth backends) – Django
Additionally, I’ve realized we got distracted and never analyzed what needs to be asyncified in AbstractBaseUser and related functionality:
I think we’ll call that task number 7
In summary, using the original numbering:
- DONE
- DONE
- DONE
- New ticket: #35303 (Add async implementations to contrib.auth backends) – Django
- DONE
- Blocked by (4)
- TODO: analyze what needs async versions in AbstractBaseUser/BaseUserManager and what it’s pre-reqs are
2 Likes
I’ve put up Fixed #35303 -- Added async auth backends and associated functionality by bigfootjon · Pull Request #18036 · django/django · GitHub for review.
This PR combines (4), (6) and (7) because they ended up being fairly tightly entangled.
So in summary:
- DONE
- DONE
- DONE
- Pull Request pending: Fixed #35303 -- Added async auth backends and associated functionality by bigfootjon · Pull Request #18036 · django/django · GitHub
- DONE
- DONE via (4)
- DONE via (4)
So once that PR is reviewed and merged this project will be done 
I’m not sure what I’ll turn to next, but I think there’s 2 obvious directions:
- Continue working on asyncifying contrib modules
- Contribute to the asyncification of the ORM/database layer
I’m not sure what the best use of time would be, probably (2) but I’m not sure I have the expertise to approach a change like that yet.
4 Likes
Hey @bigfootjon — thanks for the continued work here. Good effort, over a long period! 
In line with my comment on the PR I’m currently a bit sceptical about extending async very much further into the contrib modules, due to the limited need/benefit, and the large amount of duplication we run into due to the function color problem.
It’s certainly more intimidating but ORM-native wrappers around async DB connections, and streaming queries, and so on strike me as something that’s going to yield much better returns. 
1 Like
I’m currently a bit sceptical about extending async very much further into the
contribmodules, due to the limited need/benefit, and the large amount of duplication we run into due to the function color problem
Yeah totally, that’s one of the things I called out at the very beginning of this process:
B. Are there proven strategies for reducing code duplication between sync and async versions of functionality in Django or in Python broadly?..
My reading of DEP009 is that the extended goal of async django is to replace all the sync code with async wrappers eventually, so I was just hacking away on that goal. I was specifically thinking about these lines in that DEP:
The overall goal is to have every single part of Django that could be blocking -that is, which is not just simple CPU-bound computation - be async-native (run in an asynchronous event loop without blocking).
This includes features like:
- Middleware
- Views
- The ORM
- Templating
- Testing
- Caching
- Form validation
- Emails
If that DEP is no longer current I’d be eager to read what has superseded it!
It’s certainly more intimidating but ORM-native wrappers around async DB connections, and streaming queries, and so on strike me as something that’s going to yield much better returns.
Yeah I just feel a bit too intimidated to get started on that right now, but maybe it is time to give it a closer look!
1 Like
That’s a great question.
With No-GIL and sub-interpreters, and several years more of seeing async in the wild, I wonder.
Fancy starting a fresh discussion from this conclusion here? (I think it merits it)
1 Like
@bigfootjon One place that we really could make progress is updating the Channels auth stack to make use of the new async interfaces in newer versions of Django. e.g. the Sessions and Authentication middleware could leverage these directly it seems. If you wanted to input there, I’m very happy to work on that with you on the Channels repo. 
1 Like
I’ve posted a new thread: Is DEP009 ("async-capable Django") still relevant?
1 Like
Thanks @bigfootjon, good write up! 
1 Like
Aaaand here’s the issue against channels to propose the new feature: Use Django async-native APIs where possible · Issue #2089 · django/channels · GitHub
1 Like
As of today #35303 (Add async implementations to contrib.auth backends) – Django has been merged in and closed. This project is done!
3 Likes
Great work throughout @bigfootjon!
1 Like