dask / dask/distributed

Scheduler file doesn't like paths with ~

Open
#5,072 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

There are a few issues related to using `~` in the scheduler file option.

1. When starting a scheduler with a scheduler file you get a `FileNotFoundError`.

```python-traceback
$ dask-scheduler --scheduler-file "~/scheduler.json"
distributed.scheduler - INFO - -----------------------------------------------
distributed.scheduler - INFO - -----------------------------------------------
distributed.scheduler - INFO - Clear task state
distributed.scheduler - INFO - Scheduler at: tcp://10.51.100.15:8786
distributed.scheduler - INFO - dashboard at: :8787
distributed.scheduler - INFO - End scheduler at 'tcp://10.51.100.15:8786'
Traceback (most recent call last):
File "/Users/jtomlinson/miniconda3/envs/dask/bin/dask-scheduler", line 33, in
sys.exit(load_entry_point('distributed', 'console_scripts', 'dask-scheduler')())
File "/Users/jtomlinson/Projects/dask/distributed/distributed/cli/dask_scheduler.py", line 217, in go
main()
File "/Users/jtomlinson/miniconda3/envs/dask/lib/python3.8/site-packages/click/core.py", line 1137, in __call__
return self.main(*args, **kwargs)
File "/Users/jtomlinson/miniconda3/envs/dask/lib/python3.8/site-packages/click/core.py", line 1062, in main
rv = self.invoke(ctx)
File "/Users/jtomlinson/miniconda3/envs/dask/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/jtomlinson/miniconda3/envs/dask/lib/python3.8/site-packages/click/core.py", line 763, in invoke
return __callback(*args, **kwargs)
File "/Users/jtomlinson/Projects/dask/distributed/distributed/cli/dask_scheduler.py", line 208, in main
loop.run_sync(run)
File "/Users/jtomlinson/miniconda3/envs/dask/lib/python3.8/site-packages/tornado/ioloop.py", line 530, in run_sync
return future_cell[0].result()
File "/Users/jtomlinson/Projects/dask/distributed/distributed/cli/dask_scheduler.py", line 204, in run
await scheduler
File "/Users/jtomlinson/Projects/dask/distributed/distributed/core.py", line 283, in _
await self.start()
File "/Users/jtomlinson/Projects/dask/distributed/distributed/scheduler.py", line 3791, in start
with open(self.scheduler_file, "w") as f:
FileNotFoundError: [Errno 2] No such file or directory: '~/scheduler.json'
```

2. When connecting a worker or client to a scheduler with a `~` in the scheduler file path it hangs indefinitely.

```
$ dask-worker --scheduler-file "~/scheduler.json"

Hung...
```

3. A user has also reported issues with client but I have been unable to reproduce locally. dask/dask-mpi#71

```python
In [1]: from dask.distributed import Client
cl
In [2]: client = Client(scheduler_file="~/scheduler.json")

In [3]: client
Out[3]:
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.