apache / apache/airflow

on_task_instance_up_for_retry

Open
#72,426 1 comment 0 reactions 0 assignees View on GitHub
kind:feature needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Description

`on_task_instance_failed` fires identically whether a task instance is about to be automatically
retried (state set to `UP_FOR_RETRY`) or has permanently failed (state set to `FAILED`). The hook
call in `models/taskinstance.py` sits after the retry-eligibility branch and fires unconditionally
either way, using the same hook name.

### Use case/motivation

This makes the listener API noisy for a common use case: an alerting/paging integration built on
`on_task_instance_failed` gets triggered on every transient retry of a flaky task, not just the
final terminal failure, which undermines the signal-to-noise of building alerting on this hook.

Airflow's listener spec already carves out sub-outcomes this way elsewhere —
`on_task_instance_skipped` exists specifically to distinguish "skipped" from the generic
success/failure buckets. Same pattern fits here: a new `on_task_instance_up_for_retry` hookspec,
fired when a task instance is set to `UP_FOR_RETRY`, additive alongside the existing
`on_task_instance_failed` (left unchanged, to avoid a breaking change for existing listeners).

### Related issues

Was previously opened as #72379 (closed to re-open via this form so template labels apply
correctly).

### Are you willing to submit a PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start in models/taskinstance.py at the retry-eligibility branch and its current unconditional on_task_instance_failed call. Trace the listener hookspec definition and related listener tests, then verify that on_task_instance_up_for_retry fires for UP_FOR_RETRY while on_task_instance_failed remains unchanged for compatibility.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.