dask / dask/distributed

`cluster_dump` roundtrips break non-stringified keys

Open
#8,540 1 comment 0 reactions 0 assignees View on GitHub
bug diagnostics
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

**Describe the issue**:

After we removed key stringification in #8083, keys can now be composite of multiple different types. Our `cluster_dump` functionality is not capable of correctly roundtripping those.

**Minimal Complete Verifiable Example**:

```python
import dask.dataframe as dd
import pandas as pd
from distributed import Client
from distributed.cluster_dump import DumpArtefact

if __name__ == "__main__":
c = Client()
df = dd.from_pandas(pd.DataFrame({"a": [1, 2, 3]}), npartitions=3)
result = df.compute()
c.dump_cluster_state(filename="dump", format="yaml")
dump = DumpArtefact.from_url("dump.yaml")
dump.scheduler_story(df.__dask_keys__()[0])
```

raises
```
Traceback (most recent call last):
File "/Users/hendrikmakait/projects/dask/distributed/foo.py", line 12, in
dump.scheduler_story(df.__dask_keys__()[0])
File "/Users/hendrikmakait/projects/dask/distributed/distributed/cluster_dump.py", line 207, in scheduler_story
for story in _scheduler_story(keys, log):
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/hendrikmakait/projects/dask/distributed/distributed/_stories.py", line 33, in scheduler_story
if t[0] in keys_or_stimuli or keys_or_stimuli.intersection(t[3])
^^^^^^^^^^^^^^^^^^^^^^^
TypeError: unhashable type: 'list'
```
For `msgpack`, the error is the same.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.