dask / dask/distributed

Leaking thread in `SpecCluster` because `LoopRunner` is never closed

Open
#6,069 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

the `LocalCluster` isn't calling `self._loop_runner.stop()` on close

I added:

```patch
--- a/distributed/utils.py
+++ b/distributed/utils.py
@@ -459,7 +459,10 @@ class LoopRunner:
finally:
done_evt.set()

- thread = threading.Thread(target=run_loop, name="IO loop")
+ thread = threading.Thread(
+ target=run_loop,
+ name=f"IO loop for {os.environ.get('PYTEST_CURRENT_TEST')}",
+ )
thread.daemon = True
thread.start()

```

which lets me see where the loops are staying alive from:

```
[<_MainThread(MainThread, started 140507058100032)>,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
]
```

_Originally posted by @graingert in https://github.com/dask/distributed/pull/6033#issuecomment-1088739426_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.