apache / apache/airflow

Add an ability to prevent DAG future runs after a failure DAG

Open
#54,456 16 comments 1 reaction 0 assignees View on GitHub
area:core kind:feature
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 7h
Merged PRs (30d)
484

Description

### Description

Add a DAG argument which when true it will prevent the start of future runs of a DAG which had a failure in the last instance.

Don't pause (disactivate) the DAG, because developers can miss it without knowing it was deactivated automatically.

depend_on_past do something similar but only work on the task level and not on the DAG level. Using depend_on_past in the DAG arguments just prevent a task to run if the previous instance of that task didn't succeed.

### Use case/motivation

Prevent execution of DAG runs of a DAG, which depend on the past runs, in case of a failure in the last run.

### Related issues

_No response_

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by tracing how DAG arguments are defined and how the scheduler decides whether a future DAG run may start. Compare the requested behavior with depend_on_past at the task level, then verify that a failed latest run prevents the next DAG run without pausing or deactivating the DAG.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.