apache / apache/airflow

Include deferral in Task Instance Duration monitor

Open
#43,822 5 comments 0 reactions 0 assignees View on GitHub
area:UI kind:feature
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Description

The Task Instance Duration monitor displays task duration, distinguishing queued duration and total duration.
However, we have a task that spends a lot of time in deferred state, and would like the monitor to display that too (in a similar fashion as the queing time).
Actually, after being deferred the task can get queued again, and I think the monitor currently shows the _last time_ the task was queued, rather than the total time the task instance spent in queued state. In fact, I guess you could consider that a bug.

### Use case/motivation

We have an external service that a task depends on. If the Airflow monitor can display deferred state duration, we would be able monitor the service's response time through the Airflow UI.

### Related issues

_No response_

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by locating the Task Instance Duration monitor and tracing how queued and total durations are calculated and displayed. Confirm how deferred periods are represented, then define completion as showing deferred duration and aggregating all queued periods rather than only the last queued interval.

Written by the indexing model from the issue text.

Assessment

Domain
observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.