Django Background Tasks

Show & Tell
1

Hi all, I wanted to introduce myself - I’m new to the community but have been working with Python for a few years and have dabbled briefly with Django in the past but am just getting back into it.
I’ve been creating a few small projects (none public yet) and one of the issues I was running in to was deciding on a background tasks framework to use. I wanted one that was easy enough to implement for offloading simple tasks (like sending emails), but still advanced enough to cater for long-running background tasks where you might want to report on the progress.
I wrote up a blog post looking at the top 3 frameworks I could find (Celery, Django-Q, Django-RQ) and created three basic Django projects with working code for each of them. I decided that Django-RQ was the best suited for my needs and I managed to develop a more advanced example project with a long-running task that reported progress via an AJAX interface.
Keen to hear your thoughts on this and if anyone has any other tips on background tasks in Django that might be useful. I’m also interested if the new Async views in Django 3.1 will be able to remove the need for a separate package to deal with background tasks.
Post here: Django Background Tasks

Cheers,
Stuart.

5 Likes
2

Nice article, thanks for the detailed comparison! I tend to use celery for my async task work with Django. I have done only basic progress reporting on tasks, usually with something like an integer field on the model that indicates what step of processing the task is currently working on. One of the best parts about using celery is that you can use flower for monitoring your entire celery application in local development and in production. I’m not sure if the other options you explored have similar utilities, but celery+flower is super helpful. I also like using redis-commander to look at the different objects that celery creates. This was helpful in understanding how queuing actually works by looking at the raw queue data.

1 Like
3

Thanks for the article!
I was actually looking to know how could I achieve in Django what Rails does with jobs!

1 Like
4

Hi Brian - thanks for the feedback and for the extra tools to look into.

Cheers.
stuart

1 Like
5

I’m not familiar at all with Rails - what is it that Jobs does that you’re trying to replicate with Django?

6

Rails jobs are used for background tasks, basically. Which is why finding this article was very good timing for me.

1 Like