apache / apache/airflow

task.queued_duration metric not emitted in Airflow 3 (all executors)

Open
#66,067 1 comment 2 reactions 0 assignees View on GitHub
area:core area:metrics kind:bug needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Under which category would you file this issue?

Airflow Core

### Apache Airflow version

3.x

### What happened and how to reproduce it?

What happened:

task.queued_duration (legacy: dag...queued_duration) is never emitted in Airflow 3 regardless of executor (LocalExecutor, CeleryExecutor, KubernetesExecutor). The companion metric task.scheduled_duration emits correctly.

No `"cannot record queued_duration"` warnings appear in logs, the metric emission code is never reached, not silently skipped.


How to reproduce:

1. Run any DAG on Airflow 3 with StatsD configured
2. Observe task.scheduled_duration (or legacy dag...scheduled_duration) appears in metrics
3. Observe task.queued_duration (or legacy dag...queued_duration) is absent

### What you think should happen instead?
task.queued_duration should be emitted each time a task transitions from QUEUED --> RUNNING, matching Airflow 2 behavior.

### Operating System

linux

### Deployment

None

### Apache Airflow Provider(s)

_No response_

### Versions of Apache Airflow Providers

_No response_

### Official Helm Chart version

Not Applicable

### Kubernetes Version

_No response_

### Helm Chart configuration

_No response_

### Docker Image customizations

_No response_

### Anything else?

- task.scheduled_duration emits correctly because the scheduler still owns the SCHEDULED --> QUEUED transition and calls emit_state_change_metric(QUEUED) directly.
- queued_dttm is correctly set when the task enters QUEUED state , so the timing data is available, it just never gets consumed for the metric.
- This looks like a regression from Airflow 2 where queued_duration was reliably emitted for all executor types.

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.