Support passing ownerReference when creating a k8s based Dask cluster - for automatic cleanup of the k8s DaskCluster resources
- Dominant language
- Python
- Stars
- 148
- Forks
- 93
- PR merge metrics
- No merged PRs in 30d
Description
When a user pod has spawned a dask-gateway cluster within the same k8s cluster, one could in theory make that cluster clean itself up by adding a `ownerReference` pointing to the jupyterhub user server represented as a k8s pod.
I think this must be implemented by having the dask-gateway client to be configured to pass the k8s Pod's unique identifier to the dask-gateway server via the REST api as part of some optional data field. The reason for having it optional, is that only the Kubernetes implementation on how to create a dask-gateway cluster will be able to understand how to use this k8s Pod's identifier to set to ownerReference field of the k8s DaskCluster resource.
Another part to solve is to make sure there is a reliable way for the k8s Pod to be aware of its owns reference. I guess that can be set to some environment variable though by fixed configuration of the k8s Pod's manifest.
In detail, I think the following command needs to be passed information via the options argument.
https://github.com/dask/dask-gateway/blob/b86e3066839c44801c11a51564091f59b705f342/dask-gateway-server/dask_gateway_server/backends/kubernetes/backend.py#L503
### Issue action points
- [ ] For @consideRatio to investigate the idea further and provide more practical details on a suggested implementation strategy
- [ ] For anyone to review the feature proposal idea so far
Contributor guide
Assessment
This issue has not been assessed yet.