I make 2 HTTP requests from my sync Django view. Is it safe to make these requests concurrently?

Using Django Forms & APIs
1

Hello,

I run Django using sync workers on gunicorn. In one of my views, I make two serial calls to a third-party service:

def view():
    response1 = s3.get_object(...) # takes 100 ms
    response2 = s3.get_object(...) # takes 100 ms
    
    # do something with results, and return
    return JsonResponse({"result": ...})

Each call takes about 100 ms, so the view waits for 200ms total. I would like to speed this up by making the two calls concurrently. This is a classic IO problem solved in Python by either (a) an async event loop or (b) threads.

In this question on stackoverflow, I found examples of people doing both of these things inside a django view. One user made an async event loop right inside the view. The async loop is instantiated, ran, and terminated inside the context of the view. Another user reported using ThreadPoolExecutor.

Aside from the fact that this looks dirty, is this safe to do inside the synchronous view? The code example is as simple as it appears. There is no shared state, no writes, no database calls. Just trying to make 2 requests concurrently to shave off 100ms of latency.

2

I’m not 100% sure that the s3 object is thread-safe, but especially if you created a function that wrapped the creation of the s3 object as well so that there was no shared state between the functions, then this seems like a really good candidate for a ThreadPoolExecutor.

I’ve personally used ThreadPoolExecutor to make database calls and writes as well, so I don’t think you need to worry too much about that, even.

The cognitive overhead of instantiating an asyncio event loop is probably not worth it in your example. If you’re just trying to do parallel work, without also trying to massively scale, I think a thread pool executor is going be a pretty good fit for what I think you’re trying to do.

1 Like