Start another process with Django

Using Django Deployment
1

I’m writing a Django stock trading app and I’m at the point at creating the Strategy model. I want to be able to configure a strategy via the web gui (<-- details regarding the model don’t matter) and then start the strategy with a click on Start. Now Django needs to start a new Python process (process, not thread, because I need a dedicated CPU core per strategy), but how can Django do that? How to start a process and let it run in (kind of) real-time in the background? There will be a data-stream from a stock broker and therefore the strategy needs to run a loop for each new value received via the data-stream. I also want to store the data in the database. The strategy also needs to communicate with the other Django models, so read and write with the Django ORM needs to be possible.

Do you know any tools, videos or articles that will point me into the right direction? :slight_smile: Tools that I already heard of:

…but as far as I understood these tools are needed for non-time-critical tasks as these are like a scheduler like e.g. the Linux cron.

Do you have any other ideas I can follow up? :slight_smile:

2

You don’t want “Django” to do or manage that.

You want to use something like Celery, or even a worker task in channels. Both options do give you access to the database using the ORM.

You want to consider the complete life-cycle of the process - once it’s started, how long is it going to run? What is going to monitor the status of that process - possibly restarting it if it should fail for some reason? How many instances of these are going to be running? Do you have a way of limiting that number of instances?

This type of “process management” is one of those things that is an extremely bad fit for your Django wsgi process. Do it external to Django.

That would be a mistaken conclusion. These tools are generally used to run tasks that will exist beyond the lifespan of the view that initiated them. It makes no assumptions regarding the timing of the tasks themselves.

3

Thanks for the answer! Looks like I missed the good functions at Celery, I will check it out :slight_smile:

You want to consider the complete life-cycle of the process - once it’s started, how long is it going to run? What is going to monitor the status of that process - possibly restarting it if it should fail for some reason? How many instances of these are going to be running? Do you have a way of limiting that number of instances?

That are exactly the questions I have, but still missing the answers… :joy: