dask / dask/distributed

Specify different resources per worker in the hostfile

Open
#870 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

Hi,

I'm a new dask-distributed user. I would like to setup the network using the dask-ssh command and the corresponding hostfile:

dask-ssh --hostfile hostfile.txt

Let's say the workers have different resources constraints, therefore, I would like
to specify different resources per worker in the hostfile, for instance:

10.121.0.1 --nprocs 1 --nthreads 4
10.121.0.2 --nprocs 1 --nthreads 2
10.121.0.3 --nprocs 1 --nthreads 1
10.121.0.4 --nprocs 1 --nthreads 8
10.121.0.5 --nprocs 1 --nthreads 2
10.121.0.6 --nprocs 1 --nthreads 1
10.121.0.7 --nprocs 1 --nthreads 1
10.121.0.8 --nprocs 1 --nthreads 16
10.121.0.9 --nprocs 1 --nthreads 4
10.121.0.10 --nprocs 1 --nthreads 1

Can I specify different resources per worker in the hostfile and then
call dask-ssh --hostfile hostfile.txt?

Thank you very much for your help.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.