Hi all,
I’ve released django-ox (open source), a worker backend for django.tasks that stores the queue in your existing database instead of a broker.
The docs for the Tasks framework say production systems should rely on backends that supply a
worker process and a durable queue, so that’s what this is. The part I care most about
is that enqueue() is a plain INSERT on your default connection. If you enqueue inside
transaction.atomic(), the task commits or rolls back together with your business data. No
on_commit boilerplate, and no window where an order exists without its confirmation email task
or the other way round.
What’s in the box right now:
- at-least-once execution, claimed with SELECT FOR UPDATE SKIP LOCKED on databases that have it,
and an atomic compare-and-set everywhere else (SQLite included, which is handy for small
deployments) - retries with exponential backoff, keeping the traceback of every attempt in the row
- a reaper that returns tasks from dead workers to the queue
- graceful shutdown: SIGTERM stops claiming and drains in-flight tasks
- deferred tasks (
run_after), priorities and multiple queues, per the Tasks API - recurring tasks: cron schedules declared in settings next to the backend, dispatched by the
workers themselves, so there is no separate scheduler process to keep alive. A unique
constraint on (schedule, tick) makes each tick fire once however many workers are polling.
If every worker was down when a tick passed, the latest missed tick fires once on recovery
and older ones are skipped, so a nightly job survives a bad deploy window without a
thundering herd afterwards - monitoring: a queue-stats API, an
ox_healthcommand for probes and cron alerting, and
structured log events on thedjango_oxlogger - an
ox_prunecommand for cleaning up finished rows
Nothing to learn on the producer side, it’s just the standard @task decorator and .enqueue().
Free, BSD-3. The suite is 191 tests, green on SQLite and Postgres 16 across
Django 6.0 and 6.1, Python 3.12 to 3.14.
There are benchmarks against django-tasks-db in the repo, with the methodology, the raw
per-sample data, plus a soak run: 37,802 tasks under sustained
load while workers were repeatedly SIGKILLed, with zero tasks lost.
Repo: GitHub - oxpull/django-ox: Database-backed worker for Django's Tasks framework. Transactional enqueue, retries, recurring schedules, no broker to run. · GitHub
Docs: django-ox
I’d genuinely like to hear your pov, and issues are welcome!
peace:)