kubeflow / kubeflow/mpi-operator
mpijob restarts a few hours after launcher completed.
- Dominant language
- Go
- Stars
- 535
- Forks
- 238
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 10
Description
My job definition is like this.
```
apiVersion: "kubeflow.org/v1alpha1"
kind: "MPIJob"
metadata:
name: {{ job_name }}
labels:
exp_name: {{ exp_name }}
user: {{ user_name }}
spec:
backoffLimit: 0
......
restartPolicy: Never
```
After the launcher job finished, either Failed or Succeed, all worker pods terminated normally. However, after around three hours, the whole job automatically restarts. Is this expected? Should I delete the mpijob after each run?
Contributor guide
Research direction
Start with the provided MPIJob manifest and inspect the launcher and worker pod lifecycle after completion. Reproduce the delayed restart and compare the MPIJob status with operator reconciliation behavior. Done means the expected post-completion behavior is established and the issue records whether deletion is required or the operator needs correction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- distributed-systems, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100