kubeflow / kubeflow/mpi-operator
Mixed worker configuration is not possible
- Dominant language
- Go
- Stars
- 535
- Forks
- 238
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 10
Description
# Issue
Running workers with different configuration is not possible, e.g. one with `nvidia.com/gpu` and one with `amd.com/gpu`
```yaml
Worker:
replicas: 1
template:
spec:
containers:
- image: mpioperator/tensorflow-benchmarks:latest
name: tensorflow-benchmarks
resources:
limits:
nvidia.com/gpu: 2
```
# Why is this relevant
Being able to use different GPU models or vendors is a feature users ask for.
# Solution
Maybe supporting a configuration for specific workers would solve it.
```yaml
Worker-1:
template:
spec:
containers:
- image: mpioperator/tensorflow-benchmarks:latest
name: tensorflow-benchmarks
resources:
limits:
nvidia.com/gpu: 2
Worker-2:
template:
spec:
containers:
- image: mpioperator/tensorflow-benchmarks:latest
name: tensorflow-benchmarks
resources:
limits:
amd.com/gpu: 2
```
I am happy to contribute if this feature is wanted.
Contributor guide
Assessment
This issue has not been assessed yet.