Add async support for ForwardManyToOne, ForwardOneToOne, and ReverseOneToOne queries

Django Internals Async
1

I think we should consider adding asynchronous support for one-to-one query and many-to-one query.

Now, if I want to do a many-to-one query in an asynchronous view, I need to write like this:

def foreign_key_query(restaurant):
    return restaurant.place

place = await sync_to_async(foreign_key_query)(restaurant)

But I think that the following code writing will be better:

place = await restaurant.place

What’s more, this update will lay the foundation for DRF compatible asynchronous context.

Welcome to discuss. And, if necessary, I’m willing to submit a PR about it.

2

Hi @HappyDingning — Welcome, thanks for your post.

I’m going to cc @adamchainz and @andrewgodwin on this as they’ve both been involved related discussions.

Since the first discussions on adding async, lazy fetching of related objects has been an obvious sticking point. restaurant.place triggers a DB lookup standardly, and so (as you say) you need to wrap that in a function.

The first take here has always been that you should use prefetch_related() &co in order to fetch the related Place when first fetching the Restaurant, rather than using the lazy fetch. Beyond a very basic scale you want to be doing this anyway, so not adding async support here looks like encouraging best practice…

Then there’s a typing problem (and the indicated code-smell) with adding async support to the lazy fetch as you suggest:

place = await restaurant.place

Here the return type of the related attribute isn’t a(n optional) Place but that or an awaitable.

There was a similar discussion on the Django ticket #39102 about adding async support to the request.user lookup. We had a proof-of-concept there but went for a separate auser() method to keep the usage clear. The discussion there links to a Python typing issue that ruled out different typing depending on whether await was used or not: Have different methods! was the advice coming down from there.

As such I’m not sure that adding async support for lazy related object lookups is something that’s on the table.

Kind Regards,

Carlton

3

Yes, I agree with Carlton here - Python has no way of knowing if you are inside an await or not (as it would be essentially impossible to implement, given awaitables can be passed around), and so name separation is needed to allow object access to work.

Given that, we can never implement something like restaurant.place - it would be have to be something like restaurant.aattr.place or similar, and at that point you’re probably just better off prefetching.

4

Thanks for your reply. You convinced me, name separation is indeed necessary.

I also browsed the content of Django ticket #39102, and I have a little doubt about the auser() method. Why is auser a method, not a property?

5

Hi @carltongibson and @andrewgodwin, thank you for your reply.

I understand that prefetch_related is the best practice in most cases, but I feel that maybe triggers a DB lookup standardly is a more general way. So I think we can still consider adding async support for ForwardManyToOne, ForwardOneToOne, and ReverseOneToOne queries.

The name separation is necessary. But at the same time, I think it is very difficult to distinguish by naming, because it is difficult to find a suitable name that does not have the same name as the user-defined field.

I noticed that field names must not contain “__”, so is it possible to consider using “__” for naming distinction? for example:

place = await restaurant.a__place
place = await restaurant.async__place
place = await restaurant.place__async
6

Maybe… :slightly_smiling_face: You’d have to think it through, maybe with a Proof of Concept, but…

…I feel that maybe triggers a DB lookup standardly is a more general way.

I’m not sure I buy this. Lazy lookups, whilst great when you’re getting going, are a bit of a performance foot-gun, and there’s a lot of interest in finding ways to disable them, or otherwise make it impossible to fetch objects outside of an explicit query.

As such, it’s probably a virtue that folks using async need to be a little more explicit… — the back and forth to the DB (even if we get all the way to async DB drivers) is going to impose more than a move to async will give you (I’d imagine). Given that async is new we have an opportunity to ask whether lazy fetching is something to support at all. (I think it’s great in many circumstances, but I think it’s something you’re going to have moved away from before async becomes a priority.)

:woman_shrugging:

Kind Regards,

Carlton

7

OK, thank you for the reply, looks like I need to think a bit more.