Helm deployment with multiple worker configuration profiles
- Dominant language
- Python
- Stars
- 148
- Forks
- 93
- PR merge metrics
- No merged PRs in 30d
Description
In an effort to provide cost-optimization on the cluster scaling, I've been trying to schedule worker pods on to different sized nodes. I have two nodepools: small and large. I set the default pod limits to 1vcpu and 11GB RAM (wrt RAM, two fit on a small node and four would fit on to a large node), however, I also want some users to, through the `cluster.options` configuration when starting a cluster, to request higher limits (3vcpus and 21GB RAM for a full small node or even 7vcpu and 45GB RAM for a large node).
Below is the `NodeAffinity` solution that kind of works, however, when a large enough cluster is scaled up with the default limits (1vcpu and 11GB RAM), sometimes this triggers the scale up of the larger nodes. This isn't a surprise due to the `kube-scheduler` and the non-guarantee of the `preferredDuringSchedulingIgnoredDuringExecution` preferences.
```yaml
dask-gateway:
gateway:
baskend:
worker:
extraPodConfig:
tolerations:
- key: node_toleration
value: worker
operator: Equal
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: node_label
operator: In
values:
- worker
- lgworker
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
preference:
matchExpressions:
- key: node_label
operator: In
values:
- worker
- weight: 1
preference:
matchExpressions:
- key: node_label
operator: In
values:
- lgworker
```
So, I was wondering, is it feasible to have multiple profiles for worker pods? Something that would effectively extend the cluster profile options [here](https://gateway.dask.org/cluster-options.html#cluster-profiles).
Otherwise, perhaps someone may have some better insight in how to achieve what I'm trying to accomplish.
Contributor guide
Assessment
This issue has not been assessed yet.