apache / apache/airflow

Airflow does not execute a downstream task that has status "all_done"

Open
#54,514 8 comments 0 reactions 0 assignees View on GitHub
area:core area:dynamic-task-mapping kind:bug Stale Bug Report
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 7h
Merged PRs (30d)
484

Description

### Apache Airflow version

2.11.0

### If "Other Airflow 2 version" selected, which one?

2.10-composer

### What happened?

Airflow does not execute a downstream task that has status "all_done".

Status of upstream task:
```
Overall Status failed
112 Tasks Mapped
success: 94
failed: 10
skipped: 8
```

Status of Downstream task:
```
Overall Status no status
1 Task Mapped
no_status: 1
```

### What you think should happen instead?

The downstream task should execute, regardless of the status of upstream tasks.

### How to reproduce

```python
import pendulum
import random
from airflow.decorators import dag, task
from airflow.exceptions import AirflowSkipException

@dag(
dag_id="dynamic_task_demo_clean",
start_date=pendulum.datetime(2025, 8, 14, tz="UTC"),
schedule=None,
catchup=False,
tags=["demo", "core", "dynamic-mapping"],
)
def dynamic_task_demo_clean():

@task
def generate_number_list():
return [random.randint(1, 1000) for _ in range(100)]

@task(task_id="square_num", trigger_rule="all_done")
def square_num(number: int):
possible_outcomes = ["succeed", "fail", "skip"]
chosen_outcome = random.choice(possible_outcomes)

if chosen_outcome == "succeed":
return number * number
elif chosen_outcome == "fail":
raise ValueError("This is a randomly triggered error.")
else:
raise AirflowSkipException("This task was randomly skipped.")

@task(task_id="add_seven", trigger_rule="all_done")
def add_seven(number: int):
if number is None:
return None
return number + 7

number_list = generate_number_list()
squared_numbers = square_num.expand(number=number_list)
add_seven.expand(number=squared_numbers)

dynamic_task_demo_clean()
```

### Operating System

Linux

### Versions of Apache Airflow Providers

_No response_

### Deployment

Google Cloud Composer

### Deployment details

_No response_

### Anything else?

_No response_

### Are you willing to submit PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start with the provided dynamic_task_demo_clean DAG and reproduce the interaction between dynamic task mapping and the all_done trigger rule, including failed and skipped mapped instances. Trace the scheduler behavior for the downstream mapped task; done means the downstream task executes when upstream mapped instances finish in mixed states.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.