Emit metrics when Airflow auto pauses a DAG
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 472
Description
### Description
With Airflow 3's `Max Consecutive Failed DAG Runs` config, platform teams can automatically pause DAGs that repeatedly fail, preventing unnecessary resource consumption. However, once a DAG is auto-paused, there is currently no straightforward way for DAG owners to be notified.
While richer notification mechanisms (such as callbacks) may require additional design discussion, emitting a StatsD metric when the scheduler automatically pauses a DAG would be a simple and useful first step. Users could leverage this metric to configure alerts through their existing monitoring systems and promptly notify DAG owners when their DAG has been auto-paused.
### Use case/motivation
This metric would help users detect when a DAG has been automatically paused after exceeding the configured consecutive failure threshold.
Today, Airflow emits metrics for individual DAG run failures, but those metrics do not make it easy to determine when a DAG has reached the failure threshold that triggers automatic pausing. Users would need to correlate consecutive failures themselves, which can be difficult and error-prone, especially across retries, restarts, or multiple monitoring systems.
Emitting a dedicated metric when the scheduler automatically pauses a DAG would provide a clear signal that the configured threshold has been reached. Users could use this metric to trigger alerts, create dashboards, and respond promptly when a DAG stops being scheduled due to repeated failures.
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start in the scheduler path that applies Airflow 3's Max Consecutive Failed DAG Runs setting, and compare it with the existing metrics for individual DAG run failures. Add a dedicated StatsD signal when the scheduler automatically pauses a DAG after the threshold is reached, then verify that the signal distinguishes automatic pausing from ordinary run failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100