dask / dask/distributed

Future with no references reused

Open
#1,860 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

Appears that when launching tasks with the Distributed Client, sometimes `Future`s are held onto a little too long and their results may end up being reused for subsequent tasks that have the same name. A trivial example is shown below.

Test script:

```python
# test_dist_fut.py

from dask.delayed import Delayed
import distributed

if __name__ == "__main__":
c = distributed.Client()

v = Delayed("v", {"v": 0})
v = v.persist()
print(v.compute())

del v

v = Delayed("v", {"v": 1})
v = v.persist()
print(v.compute())
```


Running script:

```bash
$ python test_dist_fut.py
0
0
```


Environment:

```yaml
name: test
channels:
- conda-forge
- defaults
dependencies:
- ca-certificates=2018.1.18=0
- click=6.7=py_1
- cloudpickle=0.5.2=py_0
- cytoolz=0.9.0.1=py36_0
- dask-core=0.17.2=py_0
- distributed=1.21.4=py36_0
- heapdict=1.0.0=py36_0
- msgpack-python=0.5.5=py36_0
- ncurses=5.9=10
- openssl=1.0.2n=0
- psutil=5.4.3=py36_0
- python=3.6.4=0
- pyyaml=3.12=py36_1
- readline=7.0=0
- six=1.11.0=py36_1
- sortedcontainers=1.5.9=py36_0
- sqlite=3.20.1=2
- tblib=1.3.2=py36_0
- tk=8.6.7=0
- toolz=0.9.0=py_0
- tornado=5.0.1=py36_1
- xz=5.2.3=0
- yaml=0.1.7=0
- zict=0.1.3=py_0
- zlib=1.2.11=0
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.