Airflow does not execute a downstream task that has status "all_done"
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 484
Description
### Apache Airflow version
2.11.0
### If "Other Airflow 2 version" selected, which one?
2.10-composer
### What happened?
Airflow does not execute a downstream task that has status "all_done".
Status of upstream task:
```
Overall Status failed
112 Tasks Mapped
success: 94
failed: 10
skipped: 8
```
Status of Downstream task:
```
Overall Status no status
1 Task Mapped
no_status: 1
```
### What you think should happen instead?
The downstream task should execute, regardless of the status of upstream tasks.
### How to reproduce
```python
import pendulum
import random
from airflow.decorators import dag, task
from airflow.exceptions import AirflowSkipException
@dag(
dag_id="dynamic_task_demo_clean",
start_date=pendulum.datetime(2025, 8, 14, tz="UTC"),
schedule=None,
catchup=False,
tags=["demo", "core", "dynamic-mapping"],
)
def dynamic_task_demo_clean():
@task
def generate_number_list():
return [random.randint(1, 1000) for _ in range(100)]
@task(task_id="square_num", trigger_rule="all_done")
def square_num(number: int):
possible_outcomes = ["succeed", "fail", "skip"]
chosen_outcome = random.choice(possible_outcomes)
if chosen_outcome == "succeed":
return number * number
elif chosen_outcome == "fail":
raise ValueError("This is a randomly triggered error.")
else:
raise AirflowSkipException("This task was randomly skipped.")
@task(task_id="add_seven", trigger_rule="all_done")
def add_seven(number: int):
if number is None:
return None
return number + 7
number_list = generate_number_list()
squared_numbers = square_num.expand(number=number_list)
add_seven.expand(number=squared_numbers)
dynamic_task_demo_clean()
```
### Operating System
Linux
### Versions of Apache Airflow Providers
_No response_
### Deployment
Google Cloud Composer
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start with the provided dynamic_task_demo_clean DAG and reproduce the interaction between dynamic task mapping and the all_done trigger rule, including failed and skipped mapped instances. Trace the scheduler behavior for the downstream mapped task; done means the downstream task executes when upstream mapped instances finish in mixed states.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100