kubeflow / kubeflow/mpi-operator

Replace the plain pod workers with Indexed Job

Open
#613 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
535
Forks
238
Avg merge
2d 3h
Merged PRs (30d)
10

Description

Part-of: #373

Currently, the mpi-operator manages the plain pod workers. However, the management mechanism is similar to kubernetes batch/job, which is a reinvention of the wheel, although I understand the batch/job didn't have all features to replace the plain pod with batch/job in the past.

Because the Indexed job supports Elastically ([Elastic Indexed job](https://kubernetes.io/docs/concepts/workloads/controllers/job/#elastic-indexed-jobs)) by default since the kubernetes v1.27, even if we replace the plain pod management with Indexed job, we can support MPIJob with elastic semantics like the horovod.

So, I would propose replacing the plain pod workers with Indexd Job after the kubernetes v1.26 (EoL: 2024-02-28) has been deprecated.

Let me know what you think. @alculquicondor @terrytangyuan

Contributor guide

Open the contributing guide

Research direction

Start with the parent issue #373 and the linked Kubernetes Indexed Job and Elastic Indexed Job documentation. Establish the migration scope, including the Kubernetes version boundary and how MPIJob elastic semantics should be preserved; the work is done when the replacement design and compatibility expectations are settled.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
devops, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.