dask / dask/distributed

What might cause a a dask graph to freeze/hang when there's only 1 worker but work with no problems when there's 2 workers or above

Open
#3,730 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

In my application where I've written a test fixture to setup a dask cluster.

I noticed that when my local cluster only had 1 worker in it. The dask graph would compute up to a point and then just hang forever with no errors. The dashboard just shows the whole graph waiting on several tasks in the middle of the whole graph.

Debugging this led to nowhere. So I just added a second worker. As soon as the cluster had 2 workers, the graph completed.

The workers have no memory limit or `--memory-limit=0` and there are no constraints on the execution.

Unfortunately the graph is quite large, and so I'm not able to replicate it in this issue.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.