Add support for automatic clearing of failed child DAG tasks in TriggerDagRunOperator
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Description
I propose an enhancement to TriggerDagRunOperator that would allow automatic clearing of failed task instances in a triggered DAG run if that DAG run previously failed.
### Use case/motivation
When using TriggerDagRunOperator in synchronous mode (i.e., wait_for_completion=True), if the triggered child DAG fails (due to a failed task), the parent task also fails. However, when the operator is cleared in the UI (e.g., to retry), the associated failed child DAG run remains unchanged.
As a result:
- The retried TriggerDagRunOperator reuses the same failed DAG run,
- The child DAG run is still in failed state → the operator fails again,
- The user must manually clear failed tasks or the entire child DAG run.
This creates unnecessary friction when using DAG chaining.
### Proposed solution:
Introduce a new optional parameter:
```
TriggerDagRunOperator(
...,
auto_clear_failed_tasks=True
)
```
When set:
- Upon execution (or possibly in the clear() method logic),
- If a matching child DAG run exists and is in failed state,
- All failed task instances (and optionally downstreams) in that DAG run are automatically cleared.
This enables retrying the parent task to also retry the corresponding child DAG.
### Benefits
- Smoother reprocessing of chained DAGs
- Reduces manual intervention
- Fits naturally with Airflow's existing retry model
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start with TriggerDagRunOperator and its existing clearing and retry behavior; the issue names no file or test. Resolve the matching child-run, failed-task/downstream scope, and execution-versus-clear-time semantics, then verify that retrying the parent handles a previously failed child run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100