LocalCluster not reaping workers with run_on_scheduler and multiprocessing-method: spawn
Open
performance
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
Running the following:
```python
from dask import distributed
def noop():
pass
def noop_on_scheduler(i):
distributed.get_worker().client.run_on_scheduler(noop)
if __name__ == '__main__':
while True:
with distributed.Client(n_workers=8, threads_per_worker=1) as client:
distributed.wait(client.map(noop_on_scheduler, range(8)))
```
I get lots of worker processes being created but not destroyed, and when I hit Ctrl-C, I get lots of warning logs like:
```
distributed.process - WARNING - reaping stray process
```
Versions:
```
- dask=0.17.2=py36_0
- dask-core=0.17.2=py36_0
- distributed=1.21.6=py36_0
- tornado=4.5.3=py36_0
```
Contributor guide
Assessment
This issue has not been assessed yet.