Present task-errors caused by scheduler in UI
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Description
Some times tasks are marked as failed by the scheduler or other internal airflow components, with the reason being known by the scheduler, but not surfaced to users. Instead the task is just marked as failed, without any indication why in the Logs-tab nor under Task instance details. Only way to figure out is for admins to go through the scheduler-logs.
Example: `{scheduler_job.py:1526} ERROR - Detected zombie job: .....`, marks the task as failed.
Here the UI looks like this, with the logs-tab being only showing an empty attempt of fetching log files from worker:

If there is a need to hide potential scheduler-secrets from users, there should at least be some generic error message saying that the task was not run to completion because of internal scheduler error. So that users know if they should debug task-logic themselves, or involve admins in debugging the infrastructure.
Looking at `taskinstance.handle_failure`-function it seem like most the code for handling such errors is already very centralized. Is there a way to attach the error there onto the taskinstance itself, for use in the UI? Or is it possible to merge the relevant log streams to show in the Logs-tab?
### Use case/motivation
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start by reading taskinstance.handle_failure and scheduler_job.py around line 1526, then trace how failed task logs and Task instance details are populated in the UI. Define whether scheduler-caused errors should appear in the Logs tab, task details, or both, while keeping sensitive scheduler information hidden. Done means users can distinguish internal scheduler failures from task-logic failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, frontend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100