`client.gather` fails with a list of lists of futures and `errors="skip"`
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
**Describe the issue**:
Using a combination of code that raises an error, `client.gather` on a list of lists of futures, and `errors="skip"`, you can raise a strange type error about the list being unhashable.
**Minimal Complete Verifiable Example**:
```python
with Client() as client():
def div(x, y):
return x / y
future = client.submit(div, 1, 0) # fails
client.gather([[future]], errors="skip")
```
The above code produces the expected error report to stdout, but then raise the `TypeError`
Full Traceback
```
2022-11-18 10:52:34,325 - distributed.worker - WARNING - Compute Failed
Key: div-5d28479f603dbadfc59e274d22dcbd23
Function: div
args: (1, 0)
kwargs: {}
Exception: "ZeroDivisionError('division by zero')"
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In [4], line 6
4 return x / y
5 future = client.submit(div, 1, 0) # fails
----> 6 client.gather([[future]], errors="skip")
File ~/mambaforge/envs/dask-dev/lib/python3.10/site-packages/distributed/client.py:2226, in Client.gather(self, futures, errors, direct, asynchronous)
2224 else:
2225 local_worker = None
-> 2226 return self.sync(
2227 self._gather,
2228 futures,
2229 errors=errors,
2230 direct=direct,
2231 local_worker=local_worker,
2232 asynchronous=asynchronous,
2233 )
File ~/mambaforge/envs/dask-dev/lib/python3.10/site-packages/distributed/utils.py:339, in SyncMethodMixin.sync(self, func, asynchronous, callback_timeout, *args, **kwargs)
337 return future
338 else:
--> 339 return sync(
340 self.loop, func, *args, callback_timeout=callback_timeout, **kwargs
341 )
File ~/mambaforge/envs/dask-dev/lib/python3.10/site-packages/distributed/utils.py:406, in sync(loop, func, callback_timeout, *args, **kwargs)
404 if error:
405 typ, exc, tb = error
--> 406 raise exc.with_traceback(tb)
407 else:
408 return result
File ~/mambaforge/envs/dask-dev/lib/python3.10/site-packages/distributed/utils.py:379, in sync..f()
377 future = asyncio.wait_for(future, callback_timeout)
378 future = asyncio.ensure_future(future)
--> 379 result = yield future
380 except Exception:
381 error = sys.exc_info()
File ~/mambaforge/envs/dask-dev/lib/python3.10/site-packages/tornado/gen.py:762, in Runner.run(self)
759 exc_info = None
761 try:
--> 762 value = future.result()
763 except Exception:
764 exc_info = sys.exc_info()
File ~/mambaforge/envs/dask-dev/lib/python3.10/site-packages/distributed/client.py:2138, in Client._gather(self, futures, errors, direct, local_worker)
2135 break
2137 if bad_data and errors == "skip" and isinstance(unpacked, list):
-> 2138 unpacked = [f for f in unpacked if f not in bad_data]
2140 data.update(response["data"])
2141 result = pack_data(unpacked, merge(data, bad_data))
File ~/mambaforge/envs/dask-dev/lib/python3.10/site-packages/distributed/client.py:2138, in (.0)
2135 break
2137 if bad_data and errors == "skip" and isinstance(unpacked, list):
-> 2138 unpacked = [f for f in unpacked if f not in bad_data]
2140 data.update(response["data"])
2141 result = pack_data(unpacked, merge(data, bad_data))
TypeError: unhashable type: 'list'
```
**Anything else we need to know?**:
Using one of two minor modifications to the `gather` line causes this to behave as expected
```python
# list of futures, not a list of lists of futures
client.gather([future], errors="skip")
```
and
```python
# using the default value for `errors`
client.gather([[future]])
```
**Environment**:
- Dask version: '2022.11.0+5.g5a52c83f'
- Python version: 3.10.6
- Operating System: Mac OS (M1 chip)
- Install method (conda, pip, source): editable source
Contributor guide
Assessment
This issue has not been assessed yet.