dask / dask/distributed

unpublish_dataset makes future unattainable

Open
#3,719 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

If an asynchronous client
1. invokes get_dataset
2. invokes unpublish_dataset, while holding a reference to the future returned by get_dataset
3. tries to retrieve the future

it gets stuck forever.
Oddly, the sync client is unaffected.

Works:
```python
import gc
import time
from distributed import LocalCluster, Client

cluster = LocalCluster()
c = Client(cluster.scheduler.address)
f = c.submit(lambda: 123)
c.publish_dataset(foo=f)
del f
gc.collect()
time.sleep(.1)
f = c.get_dataset("foo")
c.unpublish_dataset("foo")
assert f.result() == 123
```

Gets stuck on the last line:
```python
import asyncio
import gc
from distributed import LocalCluster, Client

cluster = LocalCluster()
c = await Client(cluster.scheduler.address, asynchronous=True)
f = c.submit(lambda: 123)
await c.publish_dataset(foo=f)
del f
gc.collect()
await asyncio.sleep(.1)
f = await c.get_dataset("foo")
await c.unpublish_dataset("foo")
assert await f == 123
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.