Are there good reasons for Django using synchronous connection and cursors as opposed to async ones? We are thinking of extending Django with support for async connections and are wondering if sync-only is a design decision or just something that hasn’t been implemented yet.
Bonus question:
Why does Django enforce thread-locality of connections when psycopg connections are thread-safe?
If you haven’t already the Async DEP is probably worth a read as I imagine it will answer most if not all of your questions.
Yes, thank you - that answered a lot. The only question remaining is whether or not it is possible to use atomic transactions for code that is async. Is it safe to enter an atomic block in a sync function and then start an event loop within that block? Will the asynchronous code respect the atomic transaction block? Does it work if one were to use async_to_sync?
I.e. would the following code from this post have desirable behaviour?
from django.db.transaction import Atomic
from asgiref.sync import sync_to_async
class AsyncAtomicContextManager(Atomic):
def __init__(self, using=None, savepoint=True, durable=False):
super().__init__(using, savepoint, durable)
async def __aenter__(self):
await sync_to_async(super().__enter__)()
return self
async def __aexit__(self, exc_type, exc_value, traceback):
await sync_to_async(super().__exit__)(exc_type, exc_value, traceback)
This is where I bow out as I have yet to require async in any of my Django work so far.
I am sure someone more knowledgable with answer!
(I’ve been a bit checked out of this problem space, so things may have changed)
I think theoretically this could work, but I’m not sure it’s actually supported and not guaranteed to do so. And it might not be safe:
If the only asynchronous code running in an event loop inside your entire application takes place inside an atomic block then maybe this would work (as listed above). The problem comes in when you have any other async code running at the same time. Let’s say you have the following code:
asyncio.create_task(complex_job_with_db_usage_that_yields_several_times()) # 1
async with AsyncAtomicContextManager():
exists = await Model.objects.aget(...) # 2
if not exists:
await Model.objects.acreate(...) # 3
Task 1 executes first and is scheduled on the event loop. The next time control is yielded back to the event loop it will start to run. Then async task 2 happens, and while the database is processing our query due to the await control will be handed back to the asyncio loop to find something else to do. At this point, we’re supposedly inside an atomic context but suddenly task 1 starts running again! Since Django’s internals do not support async atomic block there is nothing stopping task 1 from reading and writing data that conflicts with the actions taken inside the atomic block. Then, by the time task 3 starts running the invariant we’re trying to uphold is no longer valid.
In summary, due to the complexities involved there isn’t a solution to this problem right now.