dask / dask/distributed

Super weird bug when passing a simple dict using Futures

Open
#3,965 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

I think this is a really really bad bug, unless I'm totally misunderstanding something here.

**What happened**:
Inside the futures function, the value of a dict parameter is sometimes replaced. (I'm guessing when it collides with a value of a scattered dataset.)
Output:
```
{'anything': 'a', 'anything2': {'a': , 'b': }}
{'anything': 'a_df', 'anything2': {'a': 'a_df', 'b': 'b_df'}}
{'anything': 'b', 'anything2': {'a': , 'b': }}
{'anything': 'b_df', 'anything2': {'a': 'a_df', 'b': 'b_df'}}
{'anything': 'c', 'anything2': {'a': , 'b': }}
{'anything': 'c', 'anything2': {'a': 'a_df', 'b': 'b_df'}}
```

**What you expected to happen**:
```
{'anything': 'a', 'anything2': {'a': , 'b': }}
{'anything': 'a', 'anything2': {'a': 'a_df', 'b': 'b_df'}}
{'anything': 'b', 'anything2': {'a': , 'b': }}
{'anything': 'b', 'anything2': {'a': 'a_df', 'b': 'b_df'}}
{'anything': 'c', 'anything2': {'a': , 'b': }}
{'anything': 'c', 'anything2': {'a': 'a_df', 'b': 'b_df'}}
```

**Minimal Complete Verifiable Example**:

```python
import dask
from dask.distributed import Client, LocalCluster, Future, as_completed

cluster = dask.distributed.LocalCluster()
dask = Client(cluster)

remote_data = dask.scatter(dict(a="a_df", b="b_df"))

def remote_fn(data_fn):
return data_fn

for any_value in ["a", "b", "c"]:
data = dict(
anything=any_value,
anything2=remote_data,
)
print(data)
print(dask.submit(remote_fn, data).result())
```

**Anything else we need to know?**:

**Environment**:

- Dask version: 2.19.0
- Python version: 3.6.4
- Operating System: Mac OS X (Mojave 10.14.6)
- Install method (conda, pip, source): pip

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.