Pausing a DAG stalls already-running DagRuns instead of only blocking new ones
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Description
When a DAG is paused while it has an active (running) DagRun — e.g. one started via manual trigger before the pause, or a manual trigger on an already-paused DAG that got an initial burst of tasks dispatched — the scheduler stops scheduling further task instances for that run. The run doesn't fail and logs no error; it simply stalls indefinitely in `running` state with remaining tasks stuck in `scheduled`, `queued_when=None`, never sent to the executor.
This happens because `is_paused` is checked as a single gate that governs both "should new DagRuns be created" and "should ready task instances in existing DagRuns be scheduled." The two are conflated.
### Use case/motivation
Pause is a reasonable way to say "don't start new runs of this DAG." It's not intuitive that it also freezes runs already in flight — especially since:
- No error, warning, or DAG/task state reflects why the run stopped progressing.
- A manually triggered run against a paused DAG appears to start successfully (initial tasks execute), masking the problem until the run has been silently stuck for several minutes.
- The eventual fix (unpause) has a side effect the operator may not want: it also releases scheduling for *new* runs of that DAG, when the operator's only goal was to unstick the one in-flight run.
### Suggested change
Preserve pre-run pause semantics: `is_paused` should gate DagRun *creation*, not task-instance scheduling for DagRuns already in `running` state. A DagRun that starts (however it started) while unpaused, or that gets tasks dispatched before a pause takes effect, should be allowed to run to completion. If the DAG is still paused when the run finishes, subsequent runs are blocked as expected — pause state before and after the run is preserved; only the in-flight run is exempted.
At minimum, surfacing this in the UI/logs (e.g., a task/run banner: "this DagRun's DAG is paused; scheduling is suspended") would prevent the silent-stall confusion, even if the gating behavior itself is kept as-is by design.
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Trace the scheduler paths that use is_paused, separating DagRun creation from task-instance scheduling for DagRuns already in running state. Reproduce a paused-DAG run with remaining tasks in scheduled state, then verify that an in-flight run can complete while new runs remain blocked when the DAG is paused.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100