Solve executor state mismatch / task killed externally
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Body
Users have long been confused by log messages along the lines of "task state changed externally", or something about executor state mismatch, where the thing that emits the log is essentially saying, 🤷 , no clue what happened sorry.
E.g. see here https://airflow.apache.org/docs/apache-airflow/stable/troubleshooting.html#task-state-changed-externally
This should mostly be avoidable.
If a task was killed by external user action, why do we not have a signal of that recorded somewhere (e.g. on the TI object or in event log), and why can't the executor (or task process or whatever) _see_ that and then either not have to log such a message, or log a more helpful one that says exactly what happened.
### Committer
- [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.
Contributor guide
Research direction
Start with the linked troubleshooting entry on “task state changed externally,” then trace how the executor, task process, task-instance object, and event log represent state changes. Clarify how externally killed tasks are detected and recorded; done means the mismatch message is replaced or made more helpful with an explicit recorded cause.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100