dask / dask/distributed

Erred futures are never descoped

Open
#6,310 1 comment 0 reactions 0 assignees View on GitHub
good second issue p3
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

This test hangs on the final while loop:

```python
import weakref

@gen_cluster(client=True)
async def test_descope_erred_future(c, s, a, b):
f = c.submit(lambda: 1 / 0, key="f")
ref = weakref.ref(f)
with pytest.raises(ZeroDivisionError):
await f
del f
while ref():
await asyncio.sleep(0.01)
```
Additionally, the task never disappears from ``s.tasks``.

Replacing ``with pytest.raises(ZeroDivisionError): await f`` with
```python
try:
await f
except ZeroDivisionError:
pass
```
does not change anything.

Replacing ``with pytest.raises(ZeroDivisionError): await f`` with ``await wait(f)`` makes the issue disappear.

#### Workaround
Calling ``f.release()`` does not solve the client-side leak, but it does remove the future (and all of its graph) from the scheduler and workers.

@sjperkins I believe this may be closely related with what you've been seeing on the scheduler side?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.