apache / apache/airflow

Add "Paused" state for dag steps

Open
#69,599 4 comments 0 reactions 0 assignees View on GitHub
area:core kind:feature
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Description

Add a non-terminal task instance state `paused` that lets an operator hold a
specific task instance before it runs, and later resume it without failing it,
without clearing it, and without occupying a worker or pool slot.

Behaviour:

- `scheduled` / `queued` -> `paused` via UI, CLI or REST.
- `paused` -> `scheduled` on resume; `try_number` and XComs preserved. The
scheduler picks it up on the next loop, same as a cleared task.
- Downstream tasks stay blocked while the task is `paused`; the DAG run stays
`running`.
- `paused` does not consume worker slots or pool slots (like `deferred`).
- Optional `pause_reason`, `paused_by`, and `resume_at` fields for audit and
scheduled auto-resume.

Pausing a `running` task is **out of scope** for the initial proposal. If an
operator realises too late, they can pause downstream tasks to hold the
workflow. Support for pausing running tasks (kill vs. let-finish-and-discard)
can be considered as a follow-up.

### Use case / motivation

Enterprise schedulers (UC4/Automic, Control-M, TWS) let an operator pause an
individual step when a downstream dependency is temporarily broken (DB down,
partner SFTP unavailable, maintenance window, ad-hoc manual check). Once the
issue is fixed out-of-band, the operator resumes the step and the workflow
continues without re-running upstream work.

In Airflow today the only options are:

1. Let the task fail and clear it later — re-runs work, burns retries, fires
failure alerts, breaks deadline/SLA alerts.
2. Keep it running and burn a worker slot for hours while ops fixes the
underlying issue.
3. Mark it success manually — loses the guarantee that the step actually ran.

Existing mechanisms don't cover the ad-hoc operational case:

- `dags pause` only stops **future** DAG runs, not an individual task instance.
- `deferred` requires operator-level code changes and a trigger; it can't be
applied ad-hoc by an operator.
- HITL / `awaiting_input` must be declared at DAG-authoring time; it can't be
applied to an arbitrary task on demand.
- Zero-slot pools block new tasks from starting but don't pause ones already
scheduled/queued against a different pool.

The goal is **operational transparency**: a distinct, non-terminal state with
its own audit trail (`paused_by`, `pause_reason`) instead of overloading
`failed` + `clear` for planned operational holds.

### Related issues

_No response_

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by tracing task-instance state handling through the scheduler, UI, CLI, and REST entry points named in the proposal. Map how scheduled, queued, deferred, and cleared states affect downstream blocking and worker or pool slots; done means pause and resume preserve try_number and XComs, keep the DAG running, and support the requested audit or scheduling fields.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.