Schedulers and workers not using personal config
Open
- Dominant language
- Python
- Stars
- 69
- Forks
- 40
- PR merge metrics
- No merged PRs in 30d
Description
As far as I can tell the Dask workers and scheduler on Yarn nodes only look at /etc/dask/ for configuration. It would be nice if also files in ~/.config/dask/ are used. A better option would be to allow uploading the local Dask configuration to HDFS so no Dask config is needed on the Yarn cluster nodes.
I am trying to achieve user separation and that seems only possible by specifying a different CA for each user. I have created a feature request this purpose https://github.com/dask/distributed/issues/2347
Contributor guide
Assessment
This issue has not been assessed yet.