Allow K8s Executor to add logs for tasks that failed to launch as a worker pod
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Description
The Kube Watcher in the K8s executor keeps track of tasks that are intended to be launched as K8s pods. If something happens to these pods, such as they fail to launch before the pending timeout, the Kube Watcher marks the task as failed. Instead of just recording this in the scheduler log, as currently happens, it would be beneficial to write a log of what happened in the same location that the task log would have gone.
### Use case/motivation
If something happens to a task pod, the Airflow UI doesn' show what happened to it at all. The task logs are completely blank. One needs to check the scheduler logs to troubleshoot. But in some organizations, not everyone who has access to the task logs via the Airflow UI also has access to the scheduler logs. Also, the scheduler logs have a lot going on in them for someone who just wants to know why their task didn't run.
### Related issues
#27758 Lays the groundwork for there to be contributors to task logs besides the worker
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start with the Kube Watcher in the K8s executor and review how it currently records pod-launch failures in the scheduler log. Use issue #27758 for the task-log contributor groundwork, then verify that a task that fails to launch has useful output in the same location as its normal task log.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, python
- Domain
- backend, infrastructure
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100