Cleared backfill DAG runs get stuck with "Task is in the 'None' state"
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 472
Description
### Under which category would you file this issue?
Airflow Core
### Apache Airflow version
3.3.0
### What happened and how to reproduce it?
I cleared all tasks from a certain date onwards to re-execute my models. However, the execution stalled on a backfill dag run. It says that Dependencies are blocking the task from being executed, but the task has no dependecies — it's the first task in the DAG!
I can confirm that nothing is running as well by looking at the pool, which has 0 slots taken despite having many tasks to be executed still.
Depends on past is true for the task in question. The backfill job did run successfully once before, but not after clearing it and the tasks before and after (tasks before were re-executed correctly and are hence green).
There are no failed tasks relating to this DAG.
### What you think should happen instead?
The tasks should be executed.
### Operating System
Debian GNU/Linux 12 (bookworm)
### Deployment
Official Apache Airflow Helm Chart
### Apache Airflow Provider(s)
cncf-kubernetes
### Versions of Apache Airflow Providers
apache-airflow-providers-cncf-kubernetes==10.19.0
### Official Helm Chart version
1.22.0 (latest released)
### Kubernetes Version
v1.34.6+rke2r3
### Helm Chart configuration
Nothing seems relevant to me. If needed I'll provide additional details
### Docker Image customizations
None
### Anything else?
The cluster does require a proxy to access external resources, and thus there are `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` variables defined.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Reproduce the cleared backfill DAG run in Airflow 3.3.0 with a first task using depends-on-past, then trace the dependency-blocking message and the task's None state in Airflow Core. No source file or test is identified in the issue; done means cleared tasks resume execution instead of remaining blocked while no pool slots are used.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- helm, kubernetes, python
- Domain
- backend, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 42/100