# Asynchronous ORM

**URL:** <https://forum.djangoproject.com/t/asynchronous-orm/5925>\
**Category:** Async\
**Created:** [January 1, 2021, 9:54pm UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925 "2021-01-01T21:54:58Z")\
**Posts on this page:** 20\
**Page:** 4

<div class="post-metadata">

**Author:** ![Archmonger](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/archmonger/32/4343_2.png) [@Archmonger](https://forum.djangoproject.com/u/Archmonger)\
**Post date:** [August 28, 2021, 8:00pm UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/61 "2021-08-28T20:00:06Z")

</div>

Looks like their secret sauce is the libs they’re using

From their readme:

- SQLite (requires aiosqlite)
- PostgreSQL (requires asyncpg)
- MySQL (requires aiomysql)

---

<div class="post-metadata">

**Author:** ![LiorA1](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/liora1/32/4101_2.png) [@LiorA1](https://forum.djangoproject.com/u/LiorA1)\
**Post date:** [August 28, 2021, 8:14pm UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/62 "2021-08-28T20:14:17Z")

</div>

So its Not relevant ?

---

<div class="post-metadata">

**Author:** ![Archmonger](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/archmonger/32/4343_2.png) [@Archmonger](https://forum.djangoproject.com/u/Archmonger)\
**Post date:** [August 28, 2021, 8:31pm UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/63 "2021-08-28T20:31:35Z")

</div>

It’s relevant IMO. I think staring at the design of those libs (or even using them as dependencies) is a viable path forward for Django’s ORM.

But I’m not a part of the Django team, so it’d be best for @carltongibson to answer that.

---

<div class="post-metadata">

**Author:** ![carltongibson](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/carltongibson/32/267_2.png) [@carltongibson](https://forum.djangoproject.com/u/carltongibson)\
**Post date:** [August 29, 2021, 5:46am UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/64 "2021-08-29T05:46:26Z")

</div>

It’s all relevant. Looking at what’s out there, and seeing how they’re doing it.

It would already be feasible to pass the SQL with parameters generated by the ORM to one of the async libraries and then implement an iterator to return model instances. … (e.g Tom Christie’s databases package uses SQLAlchemy to generate the SQL but you could slot the ORMs SQL in there directly.)

The issues come up when you have lazy related lookups, which would trigger another query. How are we going to handle those (see discussion above) That’s a harder topic. But not one you couldn’t play with.

---

<div class="post-metadata">

**Author:** ![andrewgodwin](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/andrewgodwin/32/2_2.png) [@andrewgodwin](https://forum.djangoproject.com/u/andrewgodwin)\
**Post date:** [August 29, 2021, 4:27pm UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/65 "2021-08-29T16:27:48Z")

</div>

Right - I think taking inspiration from other ORMs is fine, but remember that the main problem with adding async into the Django ORM is doing it in a way that integrates with the existing design patterns and code.

If we were to design a brand new, second ORM to achieve async - well, that would be a lot easier to make properly async, but it would kind of defeat the point.

---

<div class="post-metadata">

**Author:** ![auvipy](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/auvipy/32/182_2.png) [@auvipy](https://forum.djangoproject.com/u/auvipy)\
**Post date:** [October 19, 2021, 5:23am UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/66 "2021-10-19T05:23:22Z")

</div>

How andrew, how you think switching to psychopg3 would ease the effort? [psycopg 3.0.1 documentation](https://www.psycopg.org/psycopg3/docs/) itwas released recently. also is django going to switch over it over current v2.x?

---

<div class="post-metadata">

**Author:** ![paaksing](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/paaksing/32/6590_2.png) [@paaksing](https://forum.djangoproject.com/u/paaksing)\
**Post date:** [December 4, 2021, 3:16pm UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/68 "2021-12-04T15:16:19Z")

</div>

@andrewgodwin So after seeing that Django is likely not gonna make async orm into version 4.x. I made my own package to make my life easier and hopefully a lot of others developers out there. Repo:

> **[GitHub - paaksing/asgimod: Package to make Django \*usable\* in async Python](https://github.com/paaksing/asgimod)**
>
> Package to make Django \*usable\* in async Python. Contribute to paaksing/asgimod development by creating an account on GitHub.

Disclaimer: Async managers and querysets are only wrappers around the synchronous orm:  
The package includes:

- Async model mixin ~~s~~ (fully typed), `asgimod.mixins` .
- Async managers and querysets (fully typed), `asgimod.db` .
- Typed `sync_to_async` and `async_to_sync` wrappers, `asgimod.sync`.

Documentation and API references are in the README.  
Testcases and examples are in `testapp.test` module at the root.

1. Does this support foreign relation access: YES.
2. Does this allow queryset chaining: YES.
3. Does this allow queryset iterating, slicing and indexing: YES.
4. Does this affect default model manager functionality: NO, because it’s on another classproperty `aobjects`.
5. Is everything TYPED: YES, with the only exception of function parameters specification on Python\<3.10 since PEP 612 is being released on 3.10.

* * *

Hopefully this can be helpful for the Django community and contributors, some inspiration on the async orm or maybe snap this in into your official docs to help other devs.

The docs on README may be confusing, I was lazy to write html to make it more appearing. Testcases only covers the most used queries and methods, some rare methods are not covered, if someone want to PR the uncovered test cases, feel free to do so.

* * *

Also a quick explanation of WHY I chose to not extend `models.Manager` and add the async equivalent methods using the `a` prefix as I proposed earlier in this forum: The manager has methods that returns querysets, and querysets can chain more methods and to build up the internal query. For example:

```python
await Model.objects.afilter(a=b).aorder_by("a").areverse().aeval()

```

It makes absolutely no sense to call repetitively `a` prefixed methods when you can choose the correct manager to do the job.

```python
await Model.aobjects.filter(a=b).order_by("a").reverse().eval()

```

It also wouldn’t break consistencies because `aobjects` is at the same level of `asave` and `adelete`, which by consistencies the foreign relation access, I chose to also use the `a` prefix.

* * *

This package DOES NOT modify any predefined models of Django itself, meaning that the default User model, GeoDjango models, Oracles models will need to be extended with the mixin somewhere.

---

<div class="post-metadata">

**Author:** ![paaksing](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/paaksing/32/6590_2.png) [@paaksing](https://forum.djangoproject.com/u/paaksing)\
**Post date:** [December 9, 2021, 4:31pm UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/69 "2021-12-09T16:31:16Z")

</div>

@andrewgodwin I have tested this package on my own production projects, at least for the queries the project is using. Do you have any opinions on this ? Maybe I should do a separated post dedicated for this package.

---

<div class="post-metadata">

**Author:** ![Archmonger](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/archmonger/32/4343_2.png) [@Archmonger](https://forum.djangoproject.com/u/Archmonger)\
**Post date:** [December 27, 2021, 11:19am UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/70 "2021-12-27T11:19:36Z")

</div>

I happened to stumble upon another async Python ORM today and figured it’d be worth linking here for reference.

> **[GitHub - encode/orm: An async ORM. 🗃](https://github.com/encode/orm)**
>
> An async ORM. 🗃. Contribute to encode/orm development by creating an account on GitHub.

---

<div class="post-metadata">

**Author:** ![vhtkrk](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/vhtkrk/32/7271_2.png) [@vhtkrk](https://forum.djangoproject.com/u/vhtkrk)\
**Post date:** [January 31, 2022, 1:14pm UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/71 "2022-01-31T13:14:19Z")

</div>

There’s also [ormar](https://github.com/collerek/ormar) if we’re collecting async Python ORM’s here.

---

<div class="post-metadata">

**Author:** ![markedwards](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/markedwards/32/7332_2.png) [@markedwards](https://forum.djangoproject.com/u/markedwards)\
**Post date:** [February 5, 2022, 10:00am UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/72 "2022-02-05T10:00:32Z")

</div>

Is this effort still alive? The original release target for this has long gone. Is there any update on timing?

---

<div class="post-metadata">

**Author:** ![andrewgodwin](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/andrewgodwin/32/2_2.png) [@andrewgodwin](https://forum.djangoproject.com/u/andrewgodwin)\
**Post date:** [February 5, 2022, 4:07pm UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/73 "2022-02-05T16:07:20Z")

</div>

I am not able to work on it as long as the pandemic is ongoing, since pretty much all my mental effort is being used up by that. Others are welcome to continue the work, or hopefully I’ll get back to it… in future?

---

<div class="post-metadata">

**Author:** ![raydeal](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/raydeal/32/7037_2.png) [@raydeal](https://forum.djangoproject.com/u/raydeal)\
**Post date:** [February 7, 2022, 8:36pm UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/74 "2022-02-07T20:36:13Z")

</div>

I like this approach.

I am not very familiar with async, but it comes to my mind another syntax version

```auto
Model.objects.filter(a=b).order_by("a").reverse().eval().dasync()

```

it is more readable at a first glance, and dasync() (from Django async) to inform that it returns async version, equivalent of

> **await** Model.aobjects.filter(a=b).order\_by(“a”).reverse().eval()

Maybe it was considered at the beginning of the project.

---

<div class="post-metadata">

**Author:** ![paaksing](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/paaksing/32/6590_2.png) [@paaksing](https://forum.djangoproject.com/u/paaksing)\
**Post date:** [February 7, 2022, 8:51pm UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/75 "2022-02-07T20:51:14Z")

</div>

Yep, I considered that when making `asgimod`, and it was originally `.aeval`, but this was dropped because I wanted ForeignKey (and other relations) access, for example a model has a OneToOneField `other`, `o.other.aeval()` doesn’t work because `o.other` would already return the object and raise exception. So all in all it was a matter of code consistency.  
Another reason was that I would need to monkey patch a custom version of the model `Manager` class, which is something that needs to be avoided.

---

<div class="post-metadata">

**Author:** ![raydeal](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/raydeal/32/7037_2.png) [@raydeal](https://forum.djangoproject.com/u/raydeal)\
**Post date:** [February 7, 2022, 9:30pm UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/76 "2022-02-07T21:30:28Z")

</div>

> Another reason was that I would need to monkey patch a custom version of the model `Manager` class, which is something that needs to be avoided.

Why does monkey patching of model manager need to be avoided?

---

<div class="post-metadata">

**Author:** ![paaksing](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/paaksing/32/6590_2.png) [@paaksing](https://forum.djangoproject.com/u/paaksing)\
**Post date:** [February 7, 2022, 9:58pm UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/77 "2022-02-07T21:58:13Z")

</div>

It’s a personal opinion, implicitely changing the behavior of the code of another module comes at the very last in my list. Others may think otherwise, which is respected aswell.

---

<div class="post-metadata">

**Author:** ![raydeal](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/raydeal/32/7037_2.png) [@raydeal](https://forum.djangoproject.com/u/raydeal)\
**Post date:** [February 15, 2022, 10:56am UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/78 "2022-02-15T10:56:24Z")

</div>

As I understand async, it is usefull with time consuming i/o operations. Processing `Model.objects.filter(a=b).order_by(“a”).reverse().eval()` is not i/o operation, it is CPU/memory operation. And until it sends statement to data base it is quite quick vs db operations. So one place where it is worth apply async is connecting to db, sending statement and waiting for result from db.  
It is worth to using profiler to see how much time takes every part of that. We need to remember that every await-async is context switching time consuming and resources (cpu, memory) consuming as well.

I was wondering if django orm would work as async orm (async data base operations only) do we need bother about sync version longer? Even if user wouldn’t use async “frontend”, orm could work asynchronously if python version supports it. It would just send database statement asynchronously.

---

<div class="post-metadata">

**Author:** ![Archmonger](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/archmonger/32/4343_2.png) [@Archmonger](https://forum.djangoproject.com/u/Archmonger)\
**Post date:** [February 15, 2022, 11:35am UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/79 "2022-02-15T11:35:03Z")

</div>

For backwards compatibility reasons. If the old ORM is immediately dropped in favor of async, then all existing code breaks. For example, any package on PyPI that makes ORM calls would immediately be broken. Same with existing Django projects.

Based on what I’ve seen over the years, the Django team has a heavy priority on long-term backwards compatibility. So in this case, there’s an active effort to avoid breaking a large number of packages.

I personally think Python async was a big enough language milestone to make breaking changes within Django, similar to how Python 2 to 3 broke the ecosystem. Since the Python foundation has said Python 4 will likely never come, that scale of breaking changes can only occur as a deliberate decision on Django’s side.

But deliberately making catastrophic/cascading breaking changes is generally hard to justify.

---

<div class="post-metadata">

**Author:** ![raydeal](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/raydeal/32/7037_2.png) [@raydeal](https://forum.djangoproject.com/u/raydeal)\
**Post date:** [February 15, 2022, 5:35pm UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/80 "2022-02-15T17:35:02Z")

</div>

Yeah, I see that psycopg3 and its [django implementation -ticket #33308](https://code.djangoproject.com/ticket/33308) is that kind of solution.

---

<div class="post-metadata">

**Author:** ![davidfstr](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/davidfstr/32/269_2.png) [@davidfstr](https://forum.djangoproject.com/u/davidfstr)\
**Post date:** [February 17, 2022, 4:14pm UTC](https://forum.djangoproject.com/t/asynchronous-orm/5925/81 "2022-02-17T16:14:33Z")

</div>

> [@Archmonger](#):
>
> If the old ORM is immediately dropped in favor of async

(Agreed:) Not gonna happen. Way too much breakage.

Anything introduced has to be backward-compatible.

> [@Archmonger](#):
>
> Python 2 to 3 broke the ecosystem

(Agreed:) The Python 2 to 3 transition was a long painful (and unintentionally severe) break that lasted well over a decade. Not something I’m looking to repeat as a part-time maintainer.

> [@Archmonger](#):
>
> But deliberately making catastrophic/cascading breaking changes is generally hard to justify.

+1

[Previous page](https://forum.djangoproject.com/t/asynchronous-orm/5925.md?page=3)

[Next page](https://forum.djangoproject.com/t/asynchronous-orm/5925.md?page=5)