apache / apache/airflow

Redo #59691: placeholder upstream map-index resolution breaks per-index expansion in mapped task groups

Open
#68,417 1 comment 1 reaction 0 assignees View on GitHub
area:core area:dynamic-task-mapping kind:bug
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

#59691 ("Fix upstream map index resolution after placeholder expansion") was reverted in #68418 because it regressed per-index trigger-rule evaluation inside mapped task groups.

**What #59691 did:** added `_should_use_post_expansion_placeholder` in `taskinstance.py`, which rewrites a downstream placeholder's upstream resolution from `map_index=-1` to `0` once the upstream has expanded.

**Why it broke main:** the same helper feeds *both* XCom/arg resolution (the case #59691 targeted) and trigger-rule dependency evaluation. For a task in a mapped task group whose upstream is in the *same* group (e.g. `divide(i) >> report_success(i)`), rewriting the summary instance (`-1`) to upstream index `0` makes it inherit that single instance's state — so when `divide(0)` fails, the downstream summary is marked `UPSTREAM_FAILED` and never expands. This regressed the #50210 fix; `test_one_failed_trigger_rule_in_mapped_task_group_is_per_index` (and the whole `Core...Serialization` job) went red on `main`.

**Acceptance criteria for a redo:**

- `test_downstream_placeholder_handles_upstream_post_expansion` (the #59691 scenario) passes, **and**
- `test_one_failed_trigger_rule_in_mapped_task_group_is_per_index` + `test_one_failed_trigger_rule_runs_on_indirect_failure_in_mapped_task_group` (the #50210 scenarios) pass.
- The placeholder→0 rewrite must not affect trigger-rule evaluation of not-yet-expanded summary instances (consider distinguishing the XCom-resolution caller, or handling the summary instance in `trigger_rule_dep.py`).

cc @SameerMesiah97

---
Drafted-by: Claude Code (Opus 4.8); reviewed by @potiuk before posting

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.