dag_processing.last_duration.<dag_file> metric missing
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 484
Description
### Apache Airflow version
Other Airflow 3 version (please specify below)
### If "Other Airflow 3 version" selected, which one?
3.0.6
### What happened?
Prior to version 3x, Airflow emitted the `dag_processing.last_duration.` metric. While working in MWAA with version 3.0.6, I noticed this metric is no longer available in CloudWatch. I verified MWAA is not excluding any metrics, so I expect the metric to be present.
For a sanity check, I enabled StatsD in Breeze to see if the metric is emitted or not. I don't see the metric in Grafana.
The other python file-level metric (dag_processing.last_run.seconds_ago.) is still emitted and visible in CloudWatch.
The metric is still listed in the [Airflow metrics documentation](https://airflow.apache.org/docs/apache-airflow/3.0.6/administration-and-deployment/logging-monitoring/metrics.html#timers).
### What you think should happen instead?
I expected the metric to be present. I find it useful when identifying python files that may need to be tuned for performance.
### How to reproduce
For MWAA, create an MWAA environment with version 3.0.6 and upload a python file (with dag) to the Amazon S3 bucket. Once parsing takes place, attempt to find the metric in Amazon CloudWatch.
### Operating System
AL2023
### Versions of Apache Airflow Providers
_No response_
### Deployment
Amazon (AWS) MWAA
### Deployment details
This is a standard MWAA deployment. I did not add/modify any of the `[metrics]` config items.
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start with the Airflow metrics documentation and reproduce the issue by enabling StatsD in Breeze, comparing the missing last_duration metric with dag_processing.last_run.seconds_ago. Trace the metric emission path and verify the result in StatsD or Grafana; done means the metric is emitted again and remains consistent with the documented timers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 39/100