dask / dask/dask-kubernetes

Autoscaling of different WorkerGroups

Open
#933 2 comments 0 reactions 0 assignees View on GitHub
enhancement help wanted
Dominant language
Python
Stars
324
Forks
157
PR merge metrics
No merged PRs in 30d

Description

Hi,

I was wondering if there is any plans for allowing to define autoscalers per worker group, I think this could be quite a useful feature.

E.g. in our case we have two groups, "low-mem" which is the default group and "high-mem", the service picks the workers based on guessed workload. Small tasks go to low-mem, big tasks go to high-mem.
We then have an autoscaler, but as the docs states, only the default group is auto scaled.

Our current solution is to have to clusters and pick between the two, but maybe I'm missing something?

Cheers

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.