Hello, I am receiving a 502 Bad Gateway error when i’m sending a request on my django application.
This request starts a process which reads a json file and put the items in the database. This takes over an hour because of the high amount of items.
The strange thing is, it works on my local device and for other files on the server as well.
However, I’m receiving a bad gateway error when I am running this import on the server.
I am using nginx but I can’t find any error logs at var/log/nginx.
Can someone help me out?
If this process is causing a view to run for more than an hour, you’re likely running into one (or more) different timeout situations.
Anything running that long shouldn’t be done directly in a view. You should use Celery (or something like it) to have that process execute outside the main stream of execution of your Django process.
Thanks for your answer. And do you think this is a possible solution for not getting a bad request error?
If found the error by the way.
2024/02/16 11:34:12 [error] 9#9: *9 upstream prematurely closed connection while reading response header from upstream, client: 213.34.28.250, server, request: “POST /import HTTP/1.1”, upstream:
I’m not sure I’m following what you’re trying to ask here.
If you’re asking if using celery is going to allow you to resolve the 502 return codes you are getting, then yes, I would expect that.
yes that was my question. is it also possible to use an async method to solve this problem? This import task is only used by the superadmin sometimes and celery is a bigger change to my appliciation.
Adding celery is a very minor change to your application - especially if you only need it for that one feature and if your alternative is making the rest of your entire project async-compatible to try to make it mostly work without using it.
Okay, so you suggest to not implement Celery in this case because this is a minor change. My question is: Is running the import function async a good solution? And is sync_to_async(importfunction()) the correct way to do this?
No, that’s not what I’m saying.
What I’m saying: Use Celery. It’s the right tool for this job.