dask / dask/dask-kubernetes

Adding a suspend field to the dask operator

Open
#701 5 comments 0 reactions 0 assignees View on GitHub
enhancement operator
Dominant language
Python
Stars
324
Forks
157
PR merge metrics
No merged PRs in 30d

Description

Kubernetes has started adding ways to add queueing capabilities into Kubernetes. The entrypoint for enabling queueing can be by implementing the suspend field.

The BatchJob API contains these field in kubernetes upstream but custom CRDs need to implement suspend schematics for queueing.

There is some work in Kueue for adding suspend capabilities to RayJob and I imagine it would be similar for this project.

Relevant PR for RayJob: https://github.com/ray-project/kuberay/pull/926
Kueue PR to incorporate RayJob: https://github.com/kubernetes-sigs/kueue/pull/667

Documentation for suspend in jobs: https://kubernetes.io/docs/concepts/workloads/controllers/job/#suspending-a-job

I think it would make sense to add it as a DaskJob but there could be a reason to implement queueing in other areas also?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.