metric: Time after triggering
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Description
Can we have a metric that indicates how long after a dag is triggered its scheduled? I notice that first task scheduling delay doesn't include external triggers but it might be nice to have how long after a dag is triggered it ran to get an idea of things like scheduler failures, full queues etc.
### Use case/motivation
We currently have a critical pipeline and a set of user focused dags that are triggered from the critical pipeline. We would like to have an idea of how fast the scheduler picks up and runs those dags. This is to get a better idea of scheduler responsiveness and also figure out if we need to increase the size of our environment, split environments
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start by locating Airflow's existing scheduler timing and metric instrumentation for task or DAG scheduling, then review how externally triggered DAG runs are recorded and picked up. Define the trigger-to-run interval and identify the relevant metric tests before implementing the new measurement. Done means the metric reflects externally triggered DAG responsiveness and is covered by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100