What might cause a a dask graph to freeze/hang when there's only 1 worker but work with no problems when there's 2 workers or above
Open
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
In my application where I've written a test fixture to setup a dask cluster.
I noticed that when my local cluster only had 1 worker in it. The dask graph would compute up to a point and then just hang forever with no errors. The dashboard just shows the whole graph waiting on several tasks in the middle of the whole graph.
Debugging this led to nowhere. So I just added a second worker. As soon as the cluster had 2 workers, the graph completed.
The workers have no memory limit or `--memory-limit=0` and there are no constraints on the execution.
Unfortunately the graph is quite large, and so I'm not able to replicate it in this issue.
Contributor guide
Assessment
This issue has not been assessed yet.