Support configuring temporary directories between scheduler and workers
Open
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
> As is this only allows for setting a single `temporary-directory` config option and it needs to uniformly apply to the scheduler, worker, and nanny. I could see that being an issue if, for example, the worker and scheduler have sufficiently different filesystems.
A workaround there would be to have separate config options for the scheduler and workers. Then parse the config option in the respective `.__init__(...)` method before forwarding it here.
_Originally posted by @jrbourbeau in https://github.com/dask/distributed/pull/7802#discussion_r1181908329_
Contributor guide
Assessment
This issue has not been assessed yet.