Hi everyone,
I opened a new-features issue proposing a small convenience method for Django’s Tasks API:
opened 03:43PM - 23 May 26 UTC
Django Core
Models/ORM
Tasks
### Code of Conduct
- [x] I agree to follow Django's Code of Conduct
### Featu… re Description
Add a convenience method to Django's built-in Tasks API for enqueueing a task only after the current database transaction commits.
Possible API:
```python
with transaction.atomic():
Thing.objects.create(num=1)
my_task.enqueue_on_commit(thing_num=1)
```
### Problem
Django's Tasks documentation already recommends wrapping task enqueueing in `transaction.on_commit()` when a task depends on database state created or updated in the current transaction:
```python
from functools import partial
from django.db import transaction
with transaction.atomic():
Thing.objects.create(num=1)
transaction.on_commit(partial(my_task.enqueue, thing_num=1))
```
This is correct, but it is verbose and easy to forget at write-to-queue boundaries. Enqueueing background work before commit is a common production race condition: the worker may start on another connection and fail to read the database row that the request just created.
In a real application using Django with Celery, every write-related enqueue boundary has to repeat this pattern:
```python
transaction.on_commit(
lambda: current_app.send_task(
TRANSCRIBE_VIDEO_TASK,
args=[video_id],
)
)
```
We use this pattern for transcription, indexing, transcript reindexing, account deletion, and evaluation tasks. Django's Tasks API could make the safe pattern explicit and discoverable.
### Request or proposal
proposal
### Additional Details
This would not change `Task.enqueue()` semantics. It would only provide a first-class API for the transaction-safe behavior Django already recommends.
The proposal is intentionally limited to the common case. `enqueue_on_commit()` would return `None`, since the real `TaskResult` is not available until the transaction commits.
This proposal also intentionally leaves out extra `transaction.on_commit()` options such as `using=` and `robust=`. Those names could conflict with normal task keyword arguments. Users who need those less common options can still call `transaction.on_commit()` directly.
An async counterpart such as `aenqueue_on_commit()` can be considered later, but the smallest useful version is the synchronous convenience method.
This is not about a specific task backend. The race exists for any backend that can run work outside the current database transaction.
### Implementation Suggestions
A minimal implementation could be:
```python
from functools import partial
from django.db import transaction
def enqueue_on_commit(self, *args, **kwargs):
transaction.on_commit(partial(self.enqueue, *args, **kwargs))
```
I'm happy to work on an implementation if the API is accepted.
The proposal is to add Task.enqueue_on_commit() as a first-class shortcut for the pattern currently documented in the Tasks framework docs:
transaction.on_commit(partial(my_task.enqueue, thing_num=1))
The goal is to make the transaction-safe enqueueing pattern easier to discover and harder to forget when a task depends on database state created or updated in the current transaction.
This would not change Task.enqueue() semantics. It would only add an explicit opt-in method:
with transaction.atomic():
Thing.objects.create(num=1)
my_task.enqueue_on_commit(thing_num=1)
The proposed method would return None, because the real TaskResult is not available until the transaction commits.
Prior art:
Celery provides delay_on_commit() and apply_async_on_commit() on DjangoTask.
The django-tasks backport/reference implementation also has transaction-aware enqueueing behavior.
I’d appreciate feedback on the API shape, especially:
Is enqueue_on_commit() the right name?
Should this remain intentionally small and omit using= / robust=?
Is returning None the least surprising behavior?
For simple support or opposition, please use reactions on the GitHub issue so the new-features process can track them cleanly.
3 Likes