Is there a time limit for processes? Best to start processes in background?

Using Django Getting Started
1

Sometimes my Django web apps will need to execute a short script with

subprocess.call([ … ]). I believe there are issues if it take more than a few seconds?

If I want to execute several of these, should I run them in the background?

Are there any issues with spawning about 10 - 20 subprocess.call’s that each

take 10-ish sec?

cs

2

Short answer - Do not do this if there’s any chance at all that the subprocess might need to continue past the point that the view returns the response. Use Celery or one of the similar background-processing task managers.

Note that even Django’s Tasks framework works under the principle of disconnecting the view from those tasks:

The actual execution must be handled by infrastructure outside Django, such as a separate process or service.

For more on this, see:

(There are a number of other topics here discussing this. Searching the forum for “background tasks” or “background processes” will give you a number of them.)

3

Celery is for enterprise level scaling for heavy traffic websites. If someone is not there yet, is it

ok to just create threads for long running functions like below?

import threading

t = threading.Thread(target=long_running_function,
                            args=args,
                            kwargs=kwargs)
t.setDaemon(True)
t.start()
4

But not only for those sites. It’s also quite suitable for small sites. (I use it regularly.)

No.

See the previous reply.

6

Ken

You were so right! I finally tried Celery and it is quite nice and easy to set up!

Thanks for the help!

Chris

7

Running several subprocesses.call() can be risky, as it can lead to timeouts or slow responses, so moving long subprocess calls to the background is the right call.