kubeflow / kubeflow/mpi-operator
Launcher has been unable to complete initialization
- Dominant language
- Go
- Stars
- 535
- Forks
- 238
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 10
Description
Hello, I tried to run the mpi-operator/examples/v1/horovod/tensorflow-mnist-elastic.yaml case, and found that the launcher has been unable to complete the Init, the log shows the following
# kubectl get pods -o wide -n mpi-operator
```
tensorflow-mnist-elastic-launcher 0/1 Init:0/1 0 6m11s
tensorflow-mnist-elastic-worker-0 1/1 Running 0 6m11s
tensorflow-mnist-elastic-worker-1 1/1 Running 0 6m11s
```
# kubectl logs tensorflow-mnist-elastic-launcher -n mpi-operator
```
Error from server (BadRequest): container "mpi-launcher" in pod "tensorflow-mnist-elastic-launcher" is waiting to start: PodInitializing
```
How can I troubleshoot or solve
Contributor guide
Research direction
The reproduction is mpi-operator/examples/v1/horovod/tensorflow-mnist-elastic.yaml; start by running it and inspecting the launcher's init container status and pod events with kubectl. Trace why the launcher remains at Init:0/1 while workers run, then verify the example reaches initialization and document the troubleshooting steps or fix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, tensorflow
- Domain
- distributed-systems, infrastructure, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100