dask / dask/dask-kubernetes

Documentation notes for dask-operator, and service account permission problems

Open
#521 1 comment 1 reaction 0 assignees View on GitHub
bug operator
Dominant language
Python
Stars
324
Forks
157
PR merge metrics
No merged PRs in 30d

Description

Thanks for the great work on the dask-operator!

Three quick notes from trying it on our cluster:
* The scaling of workers assumes that each worker pod only has one dask process. If there are multiple processes the worker-names (in dask) are something like `...-ef8274c-0`,`...-ef8274c-1` to mark which process they belong to, and scaling down will fail as the pod name will be `...-ef8274c`, so scaling down will fail. A note in the documentation that one process per pod is required would be helpful. If you have multiple processes daskworkergroup scale-up works but scale-down fails.
* If you use NodePort (as in the examples), the default service account created during the latest helm install doesn't have permission to list nodes and can lead to errors. ClusterIP works great.
* The kubeflow patch script (`kubectl patch clusterrole kubeflow-kubernetes-edit --patch '{"rules": [{"apiGroups": ["kubernetes.dask.org"],"resources": ["*"],"verbs": ["*"]}]}'`) over-writes the kubeflow permissions rather than adding the dask permissions. I'm not sure what the right way to patch-by-adding is.

Contributor guide

Open the contributing guide

Research direction

Review the documentation and NodePort examples, the Helm-installed service account behavior, and the Kubeflow patch script mentioned in the issue. Reproduce the worker scaling and permission cases if possible, then document the one-process-per-pod requirement and the observed service-account and patching behavior with a clear, verified resolution.

Written by the indexing model from the issue text.

Assessment

Tech stack
helm, kubernetes
Domain
authorization, devops, documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.