apache / apache/airflow

Operator extra links don't work per attempt

Open
#71,471 0 comments 0 reactions 0 assignees View on GitHub
affected_version:3.3 area:API area:core area:UI kind:bug
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

**TL;DR — operator extra links don't work per attempt.** After a retry, asking for an earlier
attempt's link returns the latest attempt's URL, so the button under attempt 1 opens attempt
2's logs.

It hits anyone whose link carries a job id the service mints per submission — every bundled
provider linking to per-attempt logs, so EMR, Glue, Databricks, Dataproc, Livy.

### Why

A link is one XCom row under `BaseOperatorLink.xcom_key` (`_link_` by default).

1. `XComOperatorLink.get_link` gets a `TaskInstanceKey` and ignores its `try_number`.
2. That row only ever holds the latest attempt: the run endpoint clears every XCom for the
task before each attempt ([task_instances.py#L285-L293](https://github.com/apache/airflow/blob/537feafb3697338b3988d826b9c9b45da06b1f23/airflow-core/src/airflow/api_fastapi/execution_api/routes/task_instances.py#L285-L293)),
and `XComModel.set` deletes before it inserts.

\#65661 made the endpoint resolve the requested attempt, which is what makes the wrong answer
reachable from the UI.

No workaround exists: `xcom_key` is a plain property with no access to `ti_key`, the `xcom`
table has no `try_number` column, and the clear takes every key regardless.

### Reproduce

Run a task with an extra link, make it retry, ask the extra-links endpoint for
`try_number=1`. You get attempt 2's URL, and the metastore holds one `_link_*` row.

### Fix

#71522 — each attempt's link goes to the task state store, which a retry does not clear. That
keeps the XCom clear absolute instead of carving an exemption into it, and it is where
cross-retry state already lives (#69914, #71211).

Two alternatives were drafted and closed: a row per attempt in XCom (#71518) and the same with
the link opting in (#71521). #71518 is the one to revive if this needs to reach the 3.2 line,
since the state store landed after 3.2.2.

### Apache Airflow version

3.2.2. The code path is unchanged on `main` at 537feafb3697338b3988d826b9c9b45da06b1f23.

### Are you willing to submit PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start at XComOperatorLink.get_link and task_instances.py#L285-L293, then inspect BaseOperatorLink.xcom_key and XComModel.set to understand why try_number is lost. Compare the proposed task-state-store approach in #71522; done means the extra-links endpoint returns the URL for the requested attempt while retry XCom clearing remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.