Add Task.enqueue_on_commit() to Django's Tasks API
- Dominant language
- No language data
- Stars
- 188
- Forks
- 7
- PR merge metrics
- No merged PRs in 30d
Description
### Code of Conduct
- [x] I agree to follow Django's Code of Conduct
### Feature Description
Add a convenience method to Django's built-in Tasks API for enqueueing a task only after the current database transaction commits.
Possible API:
```python
with transaction.atomic():
Thing.objects.create(num=1)
my_task.enqueue_on_commit(thing_num=1)
```
### Problem
Django's Tasks documentation already recommends wrapping task enqueueing in `transaction.on_commit()` when a task depends on database state created or updated in the current transaction:
```python
from functools import partial
from django.db import transaction
with transaction.atomic():
Thing.objects.create(num=1)
transaction.on_commit(partial(my_task.enqueue, thing_num=1))
```
This is correct, but it is verbose and easy to forget at write-to-queue boundaries. Enqueueing background work before commit is a common production race condition: the worker may start on another connection and fail to read the database row that the request just created.
In a real application using Django with Celery, every write-related enqueue boundary has to repeat this pattern:
```python
transaction.on_commit(
lambda: current_app.send_task(
TRANSCRIBE_VIDEO_TASK,
args=[video_id],
)
)
```
We use this pattern for transcription, indexing, transcript reindexing, account deletion, and evaluation tasks. Django's Tasks API could make the safe pattern explicit and discoverable.
### Request or proposal
proposal
### Additional Details
This would not change `Task.enqueue()` semantics. It would only provide a first-class API for the transaction-safe behavior Django already recommends.
The proposal is intentionally limited to the common case. `enqueue_on_commit()` would return `None`, since the real `TaskResult` is not available until the transaction commits.
This proposal also intentionally leaves out extra `transaction.on_commit()` options such as `using=` and `robust=`. Those names could conflict with normal task keyword arguments. Users who need those less common options can still call `transaction.on_commit()` directly.
An async counterpart such as `aenqueue_on_commit()` can be considered later, but the smallest useful version is the synchronous convenience method.
This is not about a specific task backend. The race exists for any backend that can run work outside the current database transaction.
### Implementation Suggestions
A minimal implementation could be:
```python
from functools import partial
from django.db import transaction
def enqueue_on_commit(self, *args, **kwargs):
transaction.on_commit(partial(self.enqueue, *args, **kwargs))
```
I'm happy to work on an implementation if the API is accepted.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.