AsyncTestClient

[evilmonkey19]

Hello everyone,

I was writing the tests for a service that I’m doing and I was trying to use the async capabilities all around, even in the tests. In some blogs I’ve seen that there’s a class which I can inherit from called AsyncTestCase. However, I could not found in the the django.testpackage.

I would love to have something as follows:

from django.test import TestCase


class MyTestCase(AsyncTestCase):
    async def setUp(self):
       self.user = await User.objects.acreate(username="user"...)
       self.async_client.login(username="user"...)
    
    async def tearDown(self):
        await self.async_client.alogout()

    async test_something(self):
        await some_async_func()

For the moment, I’m using TestCase and async_client inside, but the setUp and the tearDown are fully synchronous which beats the purpose somehow.

[adamchainz]

@evilmonkey19

In some blogs I’ve seen that there’s a class which I can inherit from called AsyncTestCase.

Where? Such a class has never existed inside Django.

---

For async setup/teardown, you can use unittest’s IsolatedAsyncioTestCase , which adds support for methods like asyncSetUp() . However, I don’t know if there are any surprises when you combine it with Django’s TestCase .

There may be potential for adding a class to Django here, to provide that support by default, but I think more research is needed by someone using async features (you? ).

[evilmonkey19]

I haven’t tried the IsolatedAsyncioTestCase as I didn’t know about. Thanks for the idea.

Myself, I’m trying to dive into the async world as its promises seem compelling. I have use case which I’ve seen throughout my several websites. Using an async worker is more than enough to handle a website with up to 10 000 users if those they do not require a lot of petitions (my case) and sometimes they need SSE. I’m really in favor of a single process serving all if it is possible as it simplifies the management and further maintainance.

I would like also to have all my tests using async, as I believe it can paralelize easier those.

[adamchainz]

async will not help scale your 10k user website, nor speed up your tests. For typical views with database usage it will be slower and harder to get right. Use it for the SSE bit only.

[evilmonkey19]

Why do you say so? I’m curious because in any other languages async is seen as a way to have threading-safe applications, and therefore speed up things while being more efficient.

[KenWhitesell]

@evilmonkey19

I’m curious because in any other languages async is seen as a way to have threading-safe applications,

I would consider this to be a mis-characterization of the benefits of async.

In a true async-first environment, everything is running in the same thread - so the sense of it being “thread-safe” is true in that you don’t have multiple threads accessing the same data objects.

This was an ideal solution for scalability in an era where CPUs and OSs could only reasonably manage a relatively limited number of threads and processes. In a modern environment, there’s much less need to put hard limits on those resources than what was needed 20 years ago.

But in any event, async does not intrinsically “speed things up”. It can increase throughput for certain types of workloads - but any individual request will generally take longer. What does happen is that you theoretically improve parallel operations by not tying up a thread while waiting for IO to complete.

Beyond that, Django isn’t an “async-first” - or even a “primarily-async” framework. All of the database operations are synchronous - the async functions are just thin wrappers around the synchronous API. Template rendering is also a synchronous operation. So using async views is causing more work in switching between these contexts.

As this type of question comes up periodically around here, you might also want to see:

• What does switching to ASGI entail?
• What is the advantage of async views? Why is adrf mentioned in Existing Third Party Packages?

[evilmonkey19]

<opinion>
I agree with you on that Django is not an “async-first” framework. However, I still think that moving towards having async in mind is not bad, actually I believe it is highly beneficial. The overhead of course is bigger, but the benefits on having non-blocking operations (if you write the code without using blocking operations and so on) can be massive at large scale. Even in the smaller scale, for instance, I have a running website in a single process which i’m using fully async views and SSE (for notifications) and I can see the benefits there. Less overhead and problem (in my case and my opinion). I use SQLite for that website and the traffic isn’t large.

Lastly, async (if done correctly) has a great advantage for debugging. As you only have one thread which move around, you can follow easier the way the code goes. Using threading this waaay more difficult as you can end-up in racing conditions, and having threading code correctly is known to be way more difficult. A more in-depth and better explained can be found in this conference.

</opinion>

You seem to know a lot about Django. Do you know if in the nearby future are there plans for this feature?

[KenWhitesell]

@evilmonkey19

Lastly, async (if done correctly) has a great advantage for debugging. As you only have one thread which move around, you can follow easier the way the code goes.

This is typically not an issue with Django, as a regular view is single-threaded by nature. Yes, there are cases where there is value to perform some degree of parallelization of processing within a view - but in my experience, it’s rare, and when I need to do it, I don’t handle it via threading. Across 12 years now of working consistently with Django, I have never encountered a problem caused by threading issues. (Race conditions are a different situation, as they can also occur within an async environment if you’re not aware of all the cases where you can yield the CPU back to the reactor.)

@evilmonkey19

The overhead of course is bigger, but the benefits on having non-blocking operations (if you write the code without using blocking operations and so on) can be massive at large scale.

This is where I think many people greatly over-estimate their needs. As I wrote in one of my posts above:

@KenWhitesell

However, I maintain that those situations do not exist for 99+% of all Django development being done. No, I don’t have any proof of that. I’m not aware of any survey or research to prove or disprove it. But the number of sites that deal with [“Facebook”, “X”, “Instagram”, “Amazon”, “Google”]-levels of traffic are exceedingly small. The number of sites that may aggregate data from multiple sources where parallel web requests or database queries would provide a tangible benefit is likely to be a much larger number - but I think that is still relatively small.

Now, if your application currently deals with this level of traffic, great! I’m happy for you and can respect the fact that you actually have this need. But you’d be in an extreme minority. On the other hand, if you’re with one of those companies with a “big idea” and that you need to “plan for the future”, I maintain you’re wasting effort. (I can name very few people in the first category. I could name hundreds of people in the latter, that ended up never needing it.)

Personally, if I had a “magic wand”, there wouldn’t be any attempt at integrating async within Django. Don’t get me wrong - I have always acknowledged that there is a need for an asynchronous framework - I just don’t think Django-as-is is the right “target”.
I think that if you’re looking to get the full benefits of async, it should be in a framework that is “async-to-the-core” (e.g. Twisted)

@evilmonkey19

You seem to know a lot about Django. Do you know if in the nearby future are there plans for this feature?

I’m sorry, I’m not sure I know which specific feature you’re asking about here.

[evilmonkey19]

I can get your point, and I agree with you that for the vast majority I wouldn’t suggest going into async as it not worth the effort. You are right that race conditions can arise as well, although are more controlled in an async environment as is single-threaded and the debugger behaves better generally.

Though I disagree that Django should not bring support for async capabilities. I believe that as one of the largest open-source project it can make a great effort (as it doing at the moment) to bring async python to a great shape. I believe there is space and time, although I am not of fundamentalist who believe async should be everywhere. It is great? Yes! But at a great cost also, and you need to be aware. The benefits can be seen at large scale but not at really small one (the 99%).

---

Going back to the topic:

The feature I’m talking about is having an AsyncTestCase which helps to have an async-first class for tests.

One way I have to do it right using pytest and pytest-django , and I think it is quite good meanwhile is the following:

import pytest

class MyTestCase:
    @pytest.fixture(autouse=True)
    async def internal_fixture(self):
        """ Combined setUp and tearDown. """
        # Do something async before
        yield
        # Do something async after
    
    async def test_something(self, internal_fixture, async_client):
        await async_client.get("/")
        # do some tests

[KenWhitesell]

@evilmonkey19

although are more controlled in an async environment as is single-threaded and the debugger behaves better generally.

This is not true in Django!

When you need to switch contexts, from async to sync (such as when making database calls) those calls are run in a separate thread from the reactor. So at a minimum, you have the reactor thread and a thread for each concurrent request making database calls.

@evilmonkey19

Though I disagree that Django should not bring support for async capabilities.

That’s not quite the intent of what I tried to say.

I would have no complaint with there being an “async Django” - having all the features of current Django except built with an async core - with the understanding that it would be a separate code base that does not affect the performance of “sync Django”. But I do feel it to be a mistake to try and layer async support in Django at the expense of sync Django.

[evilmonkey19]

When you need to switch contexts, from async to sync (such as when making database calls) those calls are run in a separate thread from the reactor. So at a minimum, you have the reactor thread and a thread for each concurrent request making database calls.

Yeah you are right that switching context isn’t the best idea. Indeed it’s a patch (hopefully temporal) at best.

But I do feel it to be a mistake to try and layer async support in Django at the expense of sync Django.

It isn’t a good idea either to do it at the expense of the sync part. However, for example, the psycopg3 project is already doing it async-first and then rewriting (automatically) to support the sync part first. It isn’t a perfect solution but it’s the most elegant I found so far. Perhaps I’m wrong, but I believe many projects that use Postgres as the DB use this driver.

[Arfey]

It’s a really funny thread

Questions: Does Django provide an AsyncTestCase class?
No. It doesn’t.

---

How can you write async tests in Django right now?

For now, the best way to test Django code is still django.test.TestCase. Under the hood, Django wraps async test methods with async_to_sync, and it kind of works.

Django relies heavily on this approach during tests, especially for:

• database transactions

• setUpTestData

• fixtures

• async_to_sync / sync_to_async

Because of this, you can’t just use IsolatedAsyncioTestCase out of the box without a lot of extra work (You can estimate the amount of work just by looking at that class: every async_to_sync call would need to be rewritten to be fully async. That’s a huge effort, and it doesn’t really make sense, because the ORM is still synchronous and runs in threads under the hood.)

So the current practical setup is:

• Use IsolatedAsyncioTestCase to test pure async code that talks to Redis, Elasticsearch, etc. (async client).

• Use django.test.TestCase for views and anything involving the ORM.

It usually looks like this:

from django.test import TestCase

class Test(TestCase):
    @classmethod
    def setUpTestData(cls):
        ...

    async def test_foo(self):
        data = foo()
        assert data == "bar"

This isn’t about test performance , but it covers about 99% of real needs.

---

BTW: TestCase creates a new event loop for each test, so you must recreate global async clients (redis, es, etc.) for every test, because they’re bound to a specific event loop.

---

About performance and async in Django

Regarding performance I don’t fully agree with all of Ken’s points, but he does make a valid argument about Django’s design. If you need to handle 10k RPCs concurrently and you’re looking for an async framework to benefit from parallelism, Django is not the right choice. Not yet.

The heavy use of async_to_sync introduces a significant performance penalty. For our asynchronous API, we’re handling about 700 rps, and we can’t push more traffic because of performance issues. To understand how bad the situation is, I switched the API from ASGI to WSGI and the 99th percentile latency improved significantly.

We rewrote most of the middleware to be fully async (we can’t rewrite all the default middleware yet), and the situation improved a bit. However, we still end up with one thread per request, which we can’t avoid because of the ORM. This severely hurts performance and increases CPU usage.

---

I don’t share the same optimism about async as you do. Async applications are also hard to debug, difficult to profile, and challenging to maintain, especially in large, high traffic projects.

Sometimes async is worth it, but without an async ORM, it’s not really about Django.

I would have no complaint with there being an “async Django” - having all the features of current Django except built with an async core - with the understanding that it would be a separate code base that does not affect the performance of “sync Django”.

@KenWhitesell Thanks for saying that 100% true

BTW if you’re interested in an async ORM, you can check out (or join) this project:
https://github.com/Arfey/django-async-backend

[evilmonkey19]

Thanks for such detailed reply I haven’t the chance to look at graphs like the ones you shared which are quite enlighting! I agree that Django isn’t async ready yet and that most project will not benefit entirely from, but I still believe the effort might be worth it. In my experience it’s easier to debug async than threading but this is perhaps my experience.

Thanks for sharing this project which I didn’t know at all. I will check it out for sure!

[Arfey]

In my experience it’s easier to debug async than threading but this is perhaps my experience.

A simple example. I have a healthcheck view for kubernetes. I don’t have heavy throttling, but the request can still take 100ms or more (99per). This means something is blocking the event loop for ~100ms (CPU-bound or synchronous I/O), which affects every request. It’s really hard to debug.

If you know an easier way to debug this, please share

def healthcheck(request):
    return JsonResponse({"status": "ok"})