dask-kubernetes-operator-role-cluster clusterrole does not have the needed ACL against pods/portforward resource
- Dominant language
- Python
- Stars
- 324
- Forks
- 157
- PR merge metrics
- No merged PRs in 30d
Description
**Describe the issue**:
The dask-kubernetes-operator pod shows an 403 Forbidden error when trying to access the k8s api. It does not seem to have the right cluster role permissions
```
[2024-10-08 21:48:24,704] httpx [INFO ] HTTP Request: GET https://10.233.0.1/api/v1/namespaces/MYNAMESPACE/pods/MYPOD/portforward?name=MYPOD&namespace=MYNAMESPACE&ports=80&_preload_content=false " HTTP/1.1 403 Forbidden"
```
Execcing into the pod and trying the same call against the API.
```
kubectl exec -it -n dask-system dask-kubernetes-operator-78d4b784cf-4r455 -- sh
$ SERVICEACCOUNT=/var/run/secrets/kubernetes.io/serviceaccount
$ NAMESPACE=$(cat ${SERVICEACCOUNT}/namespace)
$ TOKEN=$(cat ${SERVICEACCOUNT}/token)
$ CACERT=${SERVICEACCOUNT}/ca.crt
$ curl --cacert ${CACERT} --header "Authorization: Bearer ${TOKEN}" -X GET 'https://10.233.0.1/api/v1/namespaces/MYNAMESPACE/pods/MYPOD/portforward?name=MYPOD&namespace=MYNAMESPACE&ports=80&_preload_content=false'
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "pods \"MYPOD\" is forbidden: User \"system:serviceaccount:dask-system:dask-kubernetes-operator
\" cannot get resource \"pods/portforward\" in API group \"\" in the namespace \"MYNAMESPACE\"",
"reason": "Forbidden",
"details": {
"name": "MYPOD",
"kind": "pods"
},
"code": 403
}$
```
Editing the clusterrole,
```
$ kubectl edit clusterrole -n dask-system dask-kubernetes-operator-role-cluster
```
And adding
pods/portforward
Around https://github.com/dask/dask-kubernetes/blob/ab1be696d03a8963f0db120e0de993f3eda12930/dask_kubernetes/operator/deployment/helm/dask-kubernetes-operator/templates/clusterrole.yaml#L34
and restarting the application pod corrected the problem.
**Environment**:
- Dask version: dask-kubernetes-operator-2024.5.0
- Python version:
- Operating System: Rocky 8
- Install method (conda, pip, source): helm chart
Contributor guide
Assessment
This issue has not been assessed yet.