apache / apache/airflow

Solve executor state mismatch / task killed externally

Open
#44,151 0 comments 10 reactions 0 assignees View on GitHub
area:core area:logging kind:meta
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Body

Users have long been confused by log messages along the lines of "task state changed externally", or something about executor state mismatch, where the thing that emits the log is essentially saying, 🤷 , no clue what happened sorry.

E.g. see here https://airflow.apache.org/docs/apache-airflow/stable/troubleshooting.html#task-state-changed-externally

This should mostly be avoidable.

If a task was killed by external user action, why do we not have a signal of that recorded somewhere (e.g. on the TI object or in event log), and why can't the executor (or task process or whatever) _see_ that and then either not have to log such a message, or log a more helpful one that says exactly what happened.

### Committer

- [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Contributor guide

Open the contributing guide

Research direction

Start with the linked troubleshooting entry on “task state changed externally,” then trace how the executor, task process, task-instance object, and event log represent state changes. Clarify how externally killed tasks are detected and recorded; done means the mismatch message is replaced or made more helpful with an explicit recorded cause.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.