django / django/new-features

Add timeout API for tasks

Open
#144 1 comment 13 reactions 0 assignees View on GitHub
Tasks
Dominant language
No language data
Stars
188
Forks
7
PR merge metrics
No merged PRs in 30d

Description

### Code of Conduct

- [x] I agree to follow Django's Code of Conduct

### Feature Description

Tasks should support being defined with a timeout:

```python
@task(timeout=30)
def long_running_task():
time.sleep(60)
```

The above task will fail, because it takes longer than 30 seconds.

Timeouts are defined in the `task` decorator in seconds (as an `int - no fractional timeouts). They cannot be configured at enqueue time (with `.using`).

Tasks which timeout are marked as `FAILED`. Once marked as failed, they are subject to retry functionality (see #142). The exception detail will be an instance of a new `TaskTimeout` exception (which subclasses Python's `TimeoutError`).

### Problem

Currently, a task can take an undefined about of time, consuming resources for the queue worker. Usually, this is an error.

### Request or proposal

proposal

### Additional Details

_No response_

### Implementation Suggestions

I gave [a talk about some ways to implement this](https://theorangeone.net/talks/forcefully-stopping-python/) at PyCon UK, but these are out of scope of this discussion.

Tasks are given a timeout by default of 1 hour. In most cases, this is far longer than they will reasonably take, but ensures that internally there's an upper limit after which a task won't be running. This allows backends to use the timeout to indicate when a worker may have dropped a task and stop it sitting in limbo (see https://github.com/RealOrangeOne/django-tasks-db/issues/5). If a task is "RUNNING" after the timeout time (plus some padding, it is considered safe to consider failed).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.