Asynchronous ORM

Nevermind. We’re dealing with a syntax error, so the usual if sys.version_info >= (3, X) trick won’t actually prevent a syntax error on say Python 3.7 (where 7 < X). So no bare async for us.

As mentioned above, a still seems the best contender for a bare word that could also be used as a “namespace” for other async-related versions of methods/properties/etc.

Yes, this is why I was grudgingly expecting a as well if we didn’t go for the get_async, values_async naming variant instead.

Still not sure if I want to totally separate it though, given that most of the manager/queryset methods don’t need touching. It would be weird to do objects.filter(x=2).order_by("something").a.first()

I. Async Method Syntax Styles

A quick summary of syntax styles mentioned so far:

A. Trailing _async suffix:

await objects.filter(...).order_by(...).first_async()

Pros (of async): Explicit. Method overriding (ex: for save_async) feels a bit more natural.
Cons (of separate method): Encourages documenting the sync/async method pair separately, which would bloat the docs.

Pros (of trailing form): The await (at the beginning) and the async (at the end) conceptually are in a symmetic formation, which is easy to see and remember: “If you start with ‘await’ then you need to end with ‘async’.”

B1. Trailing a namespace: (new)

await objects.filter(...).order_by(...).first.a()

Pros (of a): Succinct.

B2. Leading a namespace:

await objects.filter(...).order_by(...).a.first()

Pros (of leading form): Meshes better with at least one syntax for async field access (i.e. await model.a.field). However I could see other field access syntaxes.

II. Async Field Access Syntax Styles

A. Trailing _async suffix: (new)

await choice.question_async

B1. Trailing a namespace: (new) - :no_entry_sign: Not implementable

await choice.question.a

B2. Leading a namespace:

await choice.a.question

C. Leading a lookup method: (new)

await choice.a('question')
# or
await choice.a['question']

:-1: I don’t like the extra quotes and parens which are combersome to type for a common operation.

III. Thoughts

  • I’m leaning toward the trailing forms.
  • I’m feeling a bit better about using _async as a general suffix.
    • The biggest caveat I see is possibly doubling the number of functions in the documentation. But then again, Sphinx has enough control that you can just document foo and foo_async as two method names with the same description. (The documentation for the typing module’s fields {IO, TextIO, BytesIO} shows that this pattern is possible.)
    • And of course I’m also a bit worried that _async is a lot of extra letters to type.

Comments?

I’m not too concerned about the extra letters - Django has always been about explicitness rather than conciseness, and I think that should continue to apply here.

And I agree, trailing is best, mostly as (if nothing else) it groups the sync and async versions of the function together in most views of functions and allows nice tab-completion if you have it.

@andrewgodwin Django already raises SynchronousOnlyOperation where querying from an async context. Instead of raising, could it return a proxy object with __await__ declared (or just an async def coroutine wrapper function), providing async capabilities?

This could be set up as a decorator for things like .get() and .save(), and could initially just return a sync_to_async(func) version of the method, but could also look for an async variant of the method (renamed appropriately, e.g. _async_methodname) on the object.

Another option is to use the inspect framework to walk the stack and see if we’re in a coroutine (instead of looking for the running event loop), but I’m not sure how efficient that is (or if you can even identify coroutines within inspect, but I’d expect you can). Not sure if this is worth doing, but I felt it might be worth listing.

This should allow full compatibility with the existing api, and could even potentially work for properties that weren’t collected with select_related.

(1) Okay let me then summarize the syntaxes in async context that are the best candidates so far:

  • Single-Model Returning Methods
    • await objects.filter(...).order_by(...).first_async()
    • await Question.objects.get_async(id=...)
  • Multiple-Model Returning Methods
    • choices = await question.choice_set.all_async() # need not be prefetched
    • choices = question.choice_set.all() # raises if not prefetched
  • Model Field Get
    • q = await choice.question_async
  • Model Field Set (Deferred)
    • choice.question = q
  • Single-Model Save
    • await question.save_async()
  • Multiple-Model Save
    • await choice_set.update_async(...)
    • await bulk_update_async(...)
    • await bulk_create_async(...)

(2) A user who creates their own model class which overrides save who wants their model to also be used in an async context should also override save_async:

class ProjectTextFile(models.Model):
    name = models.CharField(max_length=50)
    content = models.TextField(blank=True)

    def save(self, *args, **kwargs):
        if ProjectTextFile.is_content_too_big(self.name, self.content):
            raise ValidationError(...)
        super().save(*args, **kwargs)
    
    async def save_async(self, *args, **kwargs):
        if ProjectTextFile.is_content_too_big(self.name, self.content):
            raise ValidationError(...)
        await super().save_async(*args, **kwargs)

If only one of save or save_async is overridden, I’d have to think more about the consequences…

  • The built-in admin app currently would always use the synchronous save and ignore any save_async.
  • User code that was familiar with the model class would presumably invoke whichever save method was implemented.

(3) It feels a bit weird to have a method’s return type vary depending on whether it is being invoked from a sync context vs. an async context, but perhaps it could work…

I could see this trick working for methods like .get() and .save(). For model fields the getter would have to return a proxy if invoked from an async context, but the setter would continue to defer any actual set operation regardless of whether it is invoked from a sync/async context.


(4) Ick. Inspecting the current stack frame is almost certainly quite slow. Better stick with querying the event loop.

How much work is it to make the ORM solely async-safe? As in, just being able to safely invoke the ORM as it is today from within a coroutine, with no expectations of the coroutine yielding during database calls.

This work would (I believe) solely affect ORM internals, (eg: scoping db connections per-coroutine rather than per-thread), so could be done without being blocked by the tricky async API work you’re contemplating here.

I think it could be a useful half-way house, because many database calls are very fast and there isn’t massive gain in yielding the coroutine during that time. The big gains are with heavy queries and external API calls, where you do the extra work of using sync_to_async and async http clients respectively.

Sorry if this is slightly off topic, but I was curious what new async features (if any) are slated for Django 3.2.

We have a production Django application we’d like to add some more async to, and were just curious if there was a tentative roadmap of what new async features might land in 3.2 vs 4.0, etc. We didn’t see anything in the current 3.2 release notes yet.

Thanks.

Unfortunately, the pandemic meant that we didn’t get anything of note async-wise into Django 3.2, just some small improvements to the existing support.

2 Likes

Not a problem. Thanks for the update and for all the great work on Django. :slight_smile:

What you’re proposing doesn’t make sense to me. If the ORM does not yield during database calls then it will tie up the OS thread that is running the async caller. That will result in lower system performance than just running Django in sync mode.

Nope. Database calls go over the network. Hundreds if not thousands of times slower than regular CPU-bound calculations.

I think this is definitely an interesting idea. @andrewgodwin any thoughts?

An implementation in Django itself I think would look like:

from asyncio import get_running_loop  # Python 3.7+

class Model(...):  # django/db/models/base.py
    def save(self, *args, **kwargs):
        try:
            get_running_loop()
        except RuntimeError:  # no loop exists
            return self._save_sync(*args, **kwargs)
        else:  # loop exists
            return self._save_async(*args, **kwargs)
    
    def _save_sync(self, ...):
        ...  # the original sync implementation
    
    async def _save_async(self, ...):
        ...  # the new async implementation

And then you could have user code that looks like:

def my_endpoint_sync(request):
    first_post = Post.objects.first()
    first_post.message += '!'
    first_post.save()
    return HttpResponse()  # OK

async def my_endpoint_async(request):
    first_post = await Post.objects.first()
    first_post.message += '!'
    await first_post.save()
    return HttpResponse()  # OK

From user code this looks pretty clean. And it’s not too bad on the Django side either from what I can see.

1 Like

I agree it looks nicer on first glance, my concern is the reliability of detecting that you’re in a synchronous or asynchronous context - that code is already not 100% reliable in some very small edge cases (e.g. ipython notebooks) so we had to add a manual override to SynchronousOnlyOperation.

That, and the idea of dynamically changing the return type of a function makes me cringe a little. Just not a massive fan, I guess?

Take an existing codebase that has no async Django code, database calls are cheap business logic CRUD calls (< 0.1ms) that already block, but you also have some external API calls that take hundreds of milliseconds. It would be worthwhile turning those API calls async, but as soon as you change your request handler to async, suddenly all your ORM code now has broken and needs changing too, and since async is infectious, the change could end up more wide-reaching than you’d like.

It effectively breaks up this issue into two parts: making the ORM innards async-safe, and making the ORM API async-capable

I’ve never seen a database respond in less than a millisecond - usually I’d budget 5ms for network transit alone and then 10 - 300ms for the query depending how complex it is. That’s super dangerous because the async scheduler is assuming that you are quickly iterating through coroutines, so even blocking for 10-20 ms is going to be bad as soon as you have a couple of threads doing it.

Oh yikes. Yeah if get_running_loop() isn’t truly a reliable way to detect whether running in “an asynchronous context” then making a hard dependency on it as a global variable would be bad news.

Yes this also came to mind - I do work on mypy after all :wink: - so I was wondering if you’d bring that up. Indeed the strategy of varying the return type based on sync vs. async creates a weird return type that isn’t readily spellable.

So I guess this puts us back in the *_async boat, from my earlier comment, which is an okay place to be.


Next orders of business:

Transactions

It feels like @transaction.atomic uses thread-local state, which isn’t going to work properly in a async-scheduled green thread. I feel like there was some kind of thread-local lookalike that was designed to work in an async context, but I don’t remember what it is off the top of my head.

Database Thread Affinity

I don’t know about Postgres or MySQL, but I know for sure that the normal driver for SQLite insists that any I/O operations you send it come from a consistent single OS thread. If async Django is multiplexing all of its green threads onto one OS thread then we’re okay here. If it is not, then we could have a problem.

Heh. I was planning to have this entire thread lead into a Django Enhancement Proposal (DEP) for implementing an async ORM, but I see there already is DEP 9 (“Async-capable Django”) which even has a section for an async ORM.

And that DEP already mentions the threading problems that I just brought up, so I think I’ll drop the threading subtopic for now.

Since there already is a DEP with a lot of existing discussion, I’ll plan to next read & consider what is already there to figure out what the best next steps are to push forward an async ORM. Be back in a bit.

1 Like

(I’m planning to disconnect from this thread for a year: Too many other plates spinning. I invite other folks who are interested in Async ORM to step in and continue the discussion.)

1 Like

From a user simplicity standpoint, I still like this. Correct me if I’m wrong but under typical usage, get_running_loop() functions as intended. So I wouldn’t pull this idea off the table, it’s still an option to bug fix edge cases of it. Plus worst case scenario, the user could either override save() or use awkwardly use _save_async if they’re running some obscure edge case where they just need to get things working.

On a tangent, I’m coming here from asgrief and channels issues. I’m seeing that the Django team had turned thread_sensitive=True in response to the ORM having “subtle bugs” when it’s False, which I suppose resulted in this thread being made. But I’m struggling to track down exactly what bugs where experienced. Is there an issue list for that somewhere for me to look at? I’d like to see if I can reproduce them in a test environment.

1 Like

Unfortunately, I don’t have an exhaustive list of what caused the problems, but generally it was of the form of “persisting a connection handle to the wrong thread”.

The specific thing that made us fix it is that, if I recall correctly, AuthMiddleware gets a connection object when you go through the middleware but assigns it to a lazy property accessor, and then resolves it later in the view when you look at request.user. If you don’t thread-pin every sync function in the call stack within a request, you might end up with the middleware and the view running in different threads and, well, that varies from potentially bad (transactions not matching) to SQLite just plain not working.