apache / apache/airflow

KubernetesJobOperator task stuck in Running state when parallelism > completions

Open
#64,867 2 comments 0 reactions 0 assignees View on GitHub
area:providers kind:bug needs-triage provider:cncf-kubernetes
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Under which category would you file this issue?

Providers

### Apache Airflow version

3.1.6

### What happened and how to reproduce it?

Using a `airflow.providers.cncf.kubernetes.operators.job.KubernetesJobOperator` with `parallelism > completions` causes the airflow task to get stuck in the "Running" state.  The specified job will be launched, but regardless of success, failure, or deletion, the airflow UI says the task state is "Running".

The task logs do not print anything after:
```
INFO - Building job my-job-name
```
namely, it never prints the log:
```
INFO - Found matching pod xyz with labels abc
```
so, I'm guessing `parallelism > completions` causes issues identifying/finding the job or pods.

### What you think should happen instead?

`KubernetesJobOperator` should be able to identify/find the appropriate job and pods even if `parallelism > completions`, as it is a valid kubernetes configuration.  At the very least a warning or error should be raised so that the user is made aware why the task is stuck in a "Running" state.

### Operating System

_No response_

### Deployment

Official Apache Airflow Helm Chart

### Apache Airflow Provider(s)

cncf-kubernetes

### Versions of Apache Airflow Providers

`apache-airflow-providers-cncf-kubernetes==10.12.0`

### Official Helm Chart version

1.19.0

### Kubernetes Version

Not Applicable

### Helm Chart configuration

Not Applicable

### Docker Image customizations

Not Applicable

### Anything else?

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.