dask / dask/distributed

"Event loop is closed" in multithreaded environment

Open
#4,317 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

**What happened**:

When running the following script, often one or more of the threads will die with a `RuntimeError('Event loop is closed')`. Sometimes it completes as expected, and sometimes it'll even just hang at an arbitrary point.

**What you expected to happen**:

The script should output a bunch of log lines, and end with something like the following (but not exactly):
```
[t0] 28.910681566558363
[t2] 28.34740066976898
[t1] 31.865368307629836
None
None
None
```

**Minimal Complete Verifiable Example**:

```python
#!/usr/bin/env python3
from concurrent import futures
import random
import time

import dask
from dask.distributed import Client, LocalCluster, as_completed

def client_workload(thread_num, j, data):
stime = random.uniform(2, 4)
time.sleep(stime)
size = ""
if data is not None:
size = len(data)
print(f"[t{thread_num}] -- {j} -- {stime:.2f} -- {size}")
return stime

def thread_evtloop(thread_num):
with LocalCluster(n_workers=3) as cluster:
total = 0
with Client(cluster.scheduler_address) as client:
[data_f] = client.scatter(["some long data..."])
fts = [client.submit(client_workload, thread_num, j, data_f) for j in range(10)]

for _, result in as_completed(fts, with_results=True):
total += result

print(f"[t{thread_num}] {total}")

if __name__ == '__main__':
with futures.ThreadPoolExecutor() as e:
fts = [e.submit(thread_evtloop, i) for i in range(3)]

for d in futures.wait(fts).done:
print(d.exception())
```

**Anything else we need to know?**:

**Environment**:

- Dask version: 2.21.0, distributed version 2.30.0
- Python version: 3.7.8
- Operating System: MacOS Catalina 10.15.7
- Install method (conda, pip, source): pip

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.