Django ASGI memory leak - TLS/SSL asyncio

Hi there,
I’ve posted a question on stackoverflow week ago and I also presented what I found.
Python Django ASGI - memory leak - UPDATED #2

To sum up: even fresh Django ASGI app leaks memory. I posted steps to reproduce the problem on stackoverflow. The problem lies in asyncio and TLS/SSL.

If someone finds a configuration which doesn’t have a leak (Python version, asyncio / uvloop, daphne / uvicorn / hypercorn) please share.

1 Like

Hi! Did you find a solution for this?

Not really, I’m running Django server with malloc_trim(0) as I described in StackOverflow topic. As I keep only filename references to images in DB, all data coming in and out is JSON, all image processing is moved to AWS Lambda. This way memory usage goes up a little bit but sticks at 500-700 MB. I push some kind of an update at least once a week, as a result server starts fresh with lower memory consumption. I don’t know how things would work with 10 or 100 times higher traffic, though. As for now I can live with it and the async config works nice for me.

I’m also experiencing memory leaks, however it’s because I’m dumb and I don’t know how to code, i don’t know if i’m having leaks with the async. After i fix the memory leaks i’ll let you know if i have problems with the async itself, however, I’ll share you the configuration in order to run my server:

python -m gunicorn your-project-name.asgi:application -k uvicorn.workers.UvicornWorker --bind 0.0.0.0:8000

I’m using Django 4.2, didn’t make any changes to the asgi, will have to update it tho. What version are you using?

Did you use memory profiling in order to identify which rows are causing the leaks?

I tried I guess all config combinations for async project but it didn’t change anything regarding the problem.
I use Django 5.0 currently but also tried with 4.2 - same results.
I was using memory profiling, unfortunately this problem is deeper. It’s good for you to check if you have other memory leaks locally.
I invoked this function in suspicious places in the code to check memory usage:

def print_memory_allocation(place):
    import tracemalloc
    current, peak = tracemalloc.get_traced_memory()
    current, peak = current / 10 ** 6, peak / 10 ** 6

    print('Current and peak memory usage - {}: {} {}'.format(place, current, peak))
    snapshot = tracemalloc.take_snapshot()
    top_stats = snapshot.statistics('lineno')
    print("[ Top 5 ]")
    for stat in top_stats[:5]:
        print(stat)