I migrated a WSGI django application to ASGI and I swapped workers from
gunicorn. From the first look, the application runs fine but as I tested with more load, the requests started failing with “FATAL: sorry, too many clients already” which means that the application reached the database connections limit. As I checked the database, I found that 90% of database connections are kept around even when the application is idle.
I was able to find a similar issues being addressed in django/channels context. The closest I found was this ticket https://code.djangoproject.com/ticket/31134?cversion=2&cnum_hist=3 though in this specific case the person is using PgBouncer connection pooler and I am using a regular
The application is the same as it was before. No async code.
Is this a known problem and is there a solution for it?
Take a look at https://code.djangoproject.com/ticket/31373 and https://code.djangoproject.com/ticket/31515
What versions are you using? They’re all the latest yes? Can you try aganst 3.1a1?
If you can use the sample projects there to help us pin down an issue here that would be super.
Django is using
asgiref 3.2.7 as mentioned in those issues.
I gave it a try to Django 3.1a1. I had to get rid of DRF and a few more packages as they were incompatible with the newest Django, so I just tested this on a regular django views and there were no issues with the database connections. I ran 2000 requests with concurrency set to 100 and it was quite stable between 12-14 connections.
Almost 25% of requests failed with timeouts on a regular (sync) view. The failure percentage went down to 4% when I added async/await. Throughput stayed pretty much the same 11-12 req/s. This is by far not the best test setup.
I am keeping on a synchronous execution model until a more stable django 3.1 release.
P.S will see if I find some time to play more with a sample project.
Super. Thanks for the follow-up @egism.
I had to get rid of DRF and a few more packages as they were incompatible with the newest Django.
Issues on those projects would be welcome.
Do post again if you have time to play more.