dask / dask/distributed

client.get() on an unknown key hangs forever

Open
#3,489 8 comments 0 reactions 0 assignees View on GitHub
good first issue
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

### Issue
Calling `client.get(graph, [keys, ...])` with a key that does not exist hangs forever. The timeout argument is not respected in this case.

### Expected behavior
A clear error, the threaded scheduler throws a friendly `KeyError` for example.

### Reproducible example

```python
import operator as op
from distributed import Client
from dask import delayed

client = Client()

result = delayed(op.add)(1, 2)
graph = result.__dask_graph__()

client.get(graph, ["does_not_exist"], timeout="1s") # <--- hangs
```

Interrupting gives the following stack trace:

```python-traceback
---------------------------------------------------------------------------
KeyboardInterrupt Traceback (most recent call last)
in
1 result = delayed(op.add)(1, 2)
2 graph = result.__dask_graph__()
----> 3 client.get(graph, ["does_not_exist"], timeout="1s")

/usr/local/lib/python3.7/site-packages/distributed/client.py in get(self, dsk, keys, restrictions, loose_restrictions, resources, sync, asynchronous, direct, retries, priority, fifo_timeout, actors, **kwargs)
2590 should_rejoin = False
2591 try:
-> 2592 results = self.gather(packed, asynchronous=asynchronous, direct=direct)
2593 finally:
2594 for f in futures.values():

/usr/local/lib/python3.7/site-packages/distributed/client.py in gather(self, futures, errors, direct, asynchronous)
1888 direct=direct,
1889 local_worker=local_worker,
-> 1890 asynchronous=asynchronous,
1891 )
1892

/usr/local/lib/python3.7/site-packages/distributed/client.py in sync(self, func, asynchronous, callback_timeout, *args, **kwargs)
766 else:
767 return sync(
--> 768 self.loop, func, *args, callback_timeout=callback_timeout, **kwargs
769 )
770

/usr/local/lib/python3.7/site-packages/distributed/utils.py in sync(loop, func, callback_timeout, *args, **kwargs)
340 else:
341 while not e.is_set():
--> 342 e.wait(10)
343 if error[0]:
344 typ, exc, tb = error[0]

/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py in wait(self, timeout)
550 signaled = self._flag
551 if not signaled:
--> 552 signaled = self._cond.wait(timeout)
553 return signaled
554

/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py in wait(self, timeout)
298 else:
299 if timeout > 0:
--> 300 gotit = waiter.acquire(True, timeout)
301 else:
302 gotit = waiter.acquire(False)

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.