GKEStartJobOperator / KubernetesJobOperator fail to delete pod after it ends
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Apache Airflow Provider(s)
cncf-kubernetes
### Versions of Apache Airflow Providers
First seen in
apache-airflow-providers-cncf-kubernetes==10.1.0
Still seeing in
apache-airflow-providers-cncf-kubernetes==10.5.0
I didn't have the issue in 8.3.2
I can't test more recent versions due to versions constraints
### Apache Airflow version
2.10.5
### Operating System
cloud composer
### Deployment
Google Cloud Composer
### Deployment details
_No response_
### What happened
All my task end with
```
[2025-09-02, 15:07:22 EDT] {pod_manager.py:698} INFO - Pod PODNAME has phase Running
[2025-09-02, 15:07:24 EDT] {pod.py:1079} INFO - Skipping deleting pod: PODNAME
```
when the pod correctly succeeds in kubernetes around the same time
```
State: Terminated
Reason: Completed
Exit Code: 0
Started: Tue, 02 Sep 2025 15:05:37 -0400
Finished: Tue, 02 Sep 2025 15:07:21 -0400
```
I always use `OnFinishAction.DELETE_SUCCEEDED_POD`
### What you think should happen instead
Should delete the pod per `OnFinishAction.DELETE_SUCCEEDED_POD`
### How to reproduce
I have this on any task. I use `GKEStartPodOperator` and run python scripts through it. For example:
```
GKEStartPodOperator(
image="bash",
cmds=["/usr/local/bin/bash", "-c"],
arguments=[
"echo",
"hi",
],
)
```
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start by reproducing the pod-cleanup behavior with GKEStartPodOperator or KubernetesJobOperator and the DELETE_SUCCEEDED_POD setting, using the reported provider versions. Trace the operator and pod-manager logs around the transition from Running to Completed. Done means a successfully completed pod is deleted as requested, with coverage for the reported regression.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- gcp, kubernetes, python
- Domain
- cloud, devops
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100