Prevent stale task triggers from blocking healthy workloads
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 472
Description
### Apache Airflow version
3.3.0 and current `main`
### What happens
The triggerer supervisor loads the pinned serialized Dag for every task-associated trigger so it can determine whether the task uses `start_from_trigger`. If that serialized Dag exists but no longer contains the deferred task, `get_task()` raises `TaskNotFound` from workload construction. The exception escapes the per-batch loop, so one stale task instance prevents unrelated healthy triggers in the same batch from starting and can repeatedly restart the triggerer supervisor.
### Sanitized reproducer
1. Create two deferred task instances with ordinary triggers.
2. Leave one task instance linked to a Dag version whose serialized Dag no longer contains that task ID.
3. Leave the second task instance and its pinned Dag version valid.
4. Let the triggerer claim both trigger rows in one polling cycle.
The first call to `LazyDeserializedDAG.get_task()` raises `TaskNotFound`. The healthy trigger is not constructed during that supervisor pass.
### Expected behavior
A stale task instance must not abort construction of unrelated trigger workloads. A backportable fix can treat this narrow lookup failure as an ordinary context-free deferral, matching behavior before triggerer-side `start_from_trigger` context loading was introduced.
The durable fix should persist whether each trigger requires Dag context. Ordinary deferrals can then avoid serialized-Dag lookup entirely, direct-to-triggerer tasks can require their pinned Dag data, and legacy rows can self-classify once. If a context-required row points to a serialized Dag that lacks its task, the task instance should become `REMOVED` and unlink from the trigger without blocking healthy workloads.
### Acceptance criteria
- One stale task-associated trigger cannot abort the triggerer supervisor batch.
- Ordinary deferrals do not query or deserialize their Dag.
- Context-required triggers never run with missing or unrendered Dag context.
- Legacy rows remain safe during rolling upgrades.
---
Drafted-by: Codex (GPT-5) (no human review before posting)
Contributor guide
Research direction
Start at the triggerer supervisor's per-batch workload construction and trace LazyDeserializedDAG.get_task() through TaskNotFound handling. Verify behavior with stale and healthy trigger rows, including legacy rows and context-required triggers. Done means one stale row cannot abort the batch, ordinary deferrals avoid Dag deserialization, and missing context-required tasks are removed safely.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100