apache / apache/airflow

Cleared backfill DAG runs get stuck with "Task is in the 'None' state"

Open
#69,658 8 comments 0 reactions 0 assignees View on GitHub
area:backfill area:core area:scheduler kind:bug needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Under which category would you file this issue?

Airflow Core

### Apache Airflow version

3.3.0

### What happened and how to reproduce it?

I cleared all tasks from a certain date onwards to re-execute my models. However, the execution stalled on a backfill dag run. It says that Dependencies are blocking the task from being executed, but the task has no dependecies — it's the first task in the DAG!

Image

I can confirm that nothing is running as well by looking at the pool, which has 0 slots taken despite having many tasks to be executed still.
Depends on past is true for the task in question. The backfill job did run successfully once before, but not after clearing it and the tasks before and after (tasks before were re-executed correctly and are hence green).
There are no failed tasks relating to this DAG.

### What you think should happen instead?

The tasks should be executed.

### Operating System

Debian GNU/Linux 12 (bookworm)

### Deployment

Official Apache Airflow Helm Chart

### Apache Airflow Provider(s)

cncf-kubernetes

### Versions of Apache Airflow Providers

apache-airflow-providers-cncf-kubernetes==10.19.0

### Official Helm Chart version

1.22.0 (latest released)

### Kubernetes Version

v1.34.6+rke2r3

### Helm Chart configuration

Nothing seems relevant to me. If needed I'll provide additional details

### Docker Image customizations

None

### Anything else?

The cluster does require a proxy to access external resources, and thus there are `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` variables defined.

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Reproduce the cleared backfill DAG run in Airflow 3.3.0 with a first task using depends-on-past, then trace the dependency-blocking message and the task's None state in Airflow Core. No source file or test is identified in the issue; done means cleared tasks resume execution instead of remaining blocked while no pool slots are used.

Written by the indexing model from the issue text.

Assessment

Tech stack
helm, kubernetes, python
Domain
backend, data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.