Task instance finished with state failed, but the task instance's state attribute is queued
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 472
Description
### Under which category would you file this issue?
Providers
### Apache Airflow version
3.2.0
### What happened and how to reproduce it?
From a clean fresh install Airflow v3.2.0, after a manual (or logical date in the future, or scheduled task) task launch, it dies with the message:
```
2026-04-09T18:46:08.340118Z [info ] Task execute_workload[26ecc9e5-5612-4dc7-8987-3334f991be3b] received [celery.worker.strategy] loc=strategy.py:161
2026-04-09T18:46:08.340637Z [debug ] TaskPool: Apply (args:('execute_workload', '26ecc9e5-5612-4dc7-8987-3334f991be3b', {'argsrepr':... kwargs:{}) [celery.pool] loc=base.py:149
2026-04-09T18:46:08.341924Z [debug ] Task accepted: execute_workload[26ecc9e5-5612-4dc7-8987-3334f991be3b] pid:1034973 [celery.worker.request] loc=request.py:525
2026-04-09T18:46:08.347260Z [info ] [26ecc9e5-5612-4dc7-8987-3334f991be3b] Executing workload in Celery: dag_rel_path=PurePosixPath('AplicacionesIT/airflow-log-cleanup.py') bundle_info=BundleInfo(name='dags-folder', version=None) log_path='dag_id=airflow-log-cleanup/run_id=manual__2026-04-09T18:40:35.430296+00:00/task_id=log_cleanup_worker_num_1_dir_0/attempt=2.log' ti=TaskInstanceDTO(id=UUID('019d738b-ce7e-78fa-813d-445c019749f6'), dag_version_id=UUID('019d6e44-9c6d-7396-9815-819c356c2808'), task_id='log_cleanup_worker_num_1_dir_0', dag_id='airflow-log-cleanup', run_id='manual__2026-04-09T18:40:35.430296+00:00', try_number=2, map_index=-1, pool_slots=1, queue='default', priority_weight=1, executor_config=None, parent_context_carrier=None, context_carrier={'traceparent': '00-9d7b223ec75d27ba68df0ea91638ab60-2e2651f84e3e6e05-01'}) sentry_integration='sentry_sdk.integrations.celery.CeleryIntegration' type='ExecuteTask' [airflow.providers.celery.executors.celery_executor_utils] loc=celery_executor_utils.py:193
2026-04-09T18:46:08.349156Z [debug ] Connecting to execution API server [supervisor] loc=supervisor.py:2077 server=https://air3d01xdl.*********:9192/execution/
2026-04-09T18:46:08.350543Z [info ] Secrets backends loaded for worker [supervisor] backend_classes=['EnvironmentVariablesBackend'] count=1 loc=supervisor.py:2088
2026-04-09T18:46:08.367684Z [error ] Task execute_workload[26ecc9e5-5612-4dc7-8987-3334f991be3b] raised unexpected: psutil.NoSuchProcess(pid=1035284, msg='process PID not found') [celery.app.trace] loc=trace.py:309
Traceback (most recent call last):
File "/airflow/venv-py312/.venv/lib64/python3.12/site-packages/celery/app/trace.py", line 585, in trace_task
R = retval = fun(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^
File "/airflow/venv-py312/.venv/lib64/python3.12/site-packages/celery/app/trace.py", line 858, in __protected_call__
return self.run(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/airflow/venv-py312/.venv/lib64/python3.12/site-packages/airflow/providers/celery/executors/celery_executor_utils.py", line 202, in execute_workload
supervise(
File "/airflow/venv-py312/.venv/lib64/python3.12/site-packages/airflow/sdk/execution_time/supervisor.py", line 2097, in supervise
process = ActivitySubprocess.start(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/airflow/venv-py312/.venv/lib64/python3.12/site-packages/airflow/sdk/execution_time/supervisor.py", line 1006, in start
proc: Self = super().start(id=what.id, client=client, target=target, logger=logger, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/airflow/venv-py312/.venv/lib64/python3.12/site-packages/airflow/sdk/execution_time/supervisor.py", line 537, in start
process=psutil.Process(pid),
^^^^^^^^^^^^^^^^^^^
File "/airflow/venv-py312/.venv/lib64/python3.12/site-packages/psutil/__init__.py", line 314, in __init__
self._init(pid)
File "/airflow/venv-py312/.venv/lib64/python3.12/site-packages/psutil/__init__.py", line 360, in _init
raise NoSuchProcess(pid, msg=msg) from None
**psutil.NoSuchProcess: process PID not found** (pid=1035284)
```
### What you think should happen instead?
I think the task should run fine, as in previous Airflow 3.x or 2.x versions
### Operating System
Red Hat Enterprise Linux release 9.6 (Plow)
### Deployment
Virtualenv installation
### Apache Airflow Provider(s)
celery
### Versions of Apache Airflow Providers
package_name | description | version
=======================================+==============================================================================================+========
apache-airflow-providers-celery | Celery https://docs.celeryq.dev/en/stable/ | 3.17.1
apache-airflow-providers-common-compat | Common Compatibility Provider - providing compatibility code for previous Airflow versions | 1.14.1
apache-airflow-providers-common-io | Common IO Provider | 1.7.1
apache-airflow-providers-common-sql | Common SQL Provider https://en.wikipedia.org/wiki/SQL | 1.33.0
apache-airflow-providers-fab | Flask App Builder https://flask-appbuilder.readthedocs.io/ | 3.5.0
apache-airflow-providers-oracle | Oracle https://www.oracle.com/database/technologies/ | 4.5.1
apache-airflow-providers-postgres | PostgreSQL https://www.postgresql.org/ | 6.6.1
apache-airflow-providers-sftp | SSH File Transfer Protocol (SFTP) https://tools.ietf.org/wg/secsh/draft-ietf-secsh-filexfer/ | 5.5.0
apache-airflow-providers-smtp | Simple Mail Transfer Protocol (SMTP) https://tools.ietf.org/html/rfc5321 | 2.4.3
apache-airflow-providers-ssh | Secure Shell (SSH) https://tools.ietf.org/html/rfc4251 | 4.3.2
apache-airflow-providers-standard | Airflow Standard Provider | 1.12.1
### Official Helm Chart version
Not Applicable
### Kubernetes Version
_No response_
### Helm Chart configuration
_No response_
### Docker Image customizations
_No response_
### Anything else?
Fresh instal with celey v5.6.3, installed with
uv pip install "apache-airflow[celery,postgres,ldap,oracle,ssh,pandas]==3.2.0" --constraint "/airflow/venv-py312/constraints-3.12_320.txt"
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Assessment
This issue has not been assessed yet.