kubeflow / kubeflow/spark-operator
Support `managedBy` field on `SparkApplication` for integration with MultiKueue
- Dominant language
- Python
- Stars
- 3.2k
- Forks
- 1.5k
- Avg merge
- 5d 10h
- Merged PRs (30d)
- 13
Description
### What feature you would like to be added?
The support for the managedBy field which can delegate reconciliation from built-in controller, to a custom one.
The semantics of the field are: whenever the value is set, and it does not point to the built-in operator, then skip reconciliation
- the field is immutable
- the field is not defaulted
### Why is this needed?
Integration with [MultiKueue](https://kueue.sigs.k8s.io/docs/tasks/manage/setup_multikueue/) (multi-cluster Kueue):
- simpler installation (just follow the standard installation path), otherwise only installation of SparkApplication CRDs is required
- support for mixed setup in one cluster - some SparkApplication could be run by MultiKueue and some by the default operator
The `managedBy` field is already adopted by:
- https://github.com/kubernetes/enhancements/issues/4368 (beta in 1.32)
- https://github.com/kubernetes-sigs/jobset/issues/440 (released in 0.6)
- https://github.com/kubeflow/mpi-operator/issues/646 (released in 0.6)
- https://github.com/kubeflow/training-operator/issues/2193 (released in 1.9)
- https://github.com/ray-project/kuberay/issues/2544 (released in 1.3.0)
### Describe the solution you would like
Introduce `spec.managedBy` field in `SparkApplication`
### Describe alternatives you have considered
_No response_
### Additional context
as suggested in https://github.com/kubernetes-sigs/kueue/pull/7268#issuecomment-4063022268
### Love this feature?
Give it a 👍 We prioritize the features with most 👍
Contributor guide
Assessment
This issue has not been assessed yet.