kubeflow / kubeflow/mpi-operator

mpijob restarts a few hours after launcher completed.

Open
#86 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
535
Forks
238
Avg merge
2d 3h
Merged PRs (30d)
10

Description

My job definition is like this.
```
apiVersion: "kubeflow.org/v1alpha1"
kind: "MPIJob"
metadata:
name: {{ job_name }}
labels:
exp_name: {{ exp_name }}
user: {{ user_name }}

spec:
backoffLimit: 0
......
restartPolicy: Never

```
After the launcher job finished, either Failed or Succeed, all worker pods terminated normally. However, after around three hours, the whole job automatically restarts. Is this expected? Should I delete the mpijob after each run?

Contributor guide

Open the contributing guide

Research direction

Start with the provided MPIJob manifest and inspect the launcher and worker pod lifecycle after completion. Reproduce the delayed restart and compare the MPIJob status with operator reconciliation behavior. Done means the expected post-completion behavior is established and the issue records whether deletion is required or the operator needs correction.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
distributed-systems, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.