"Event loop is closed" in multithreaded environment
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
**What happened**:
When running the following script, often one or more of the threads will die with a `RuntimeError('Event loop is closed')`. Sometimes it completes as expected, and sometimes it'll even just hang at an arbitrary point.
**What you expected to happen**:
The script should output a bunch of log lines, and end with something like the following (but not exactly):
```
[t0] 28.910681566558363
[t2] 28.34740066976898
[t1] 31.865368307629836
None
None
None
```
**Minimal Complete Verifiable Example**:
```python
#!/usr/bin/env python3
from concurrent import futures
import random
import time
import dask
from dask.distributed import Client, LocalCluster, as_completed
def client_workload(thread_num, j, data):
stime = random.uniform(2, 4)
time.sleep(stime)
size = ""
if data is not None:
size = len(data)
print(f"[t{thread_num}] -- {j} -- {stime:.2f} -- {size}")
return stime
def thread_evtloop(thread_num):
with LocalCluster(n_workers=3) as cluster:
total = 0
with Client(cluster.scheduler_address) as client:
[data_f] = client.scatter(["some long data..."])
fts = [client.submit(client_workload, thread_num, j, data_f) for j in range(10)]
for _, result in as_completed(fts, with_results=True):
total += result
print(f"[t{thread_num}] {total}")
if __name__ == '__main__':
with futures.ThreadPoolExecutor() as e:
fts = [e.submit(thread_evtloop, i) for i in range(3)]
for d in futures.wait(fts).done:
print(d.exception())
```
**Anything else we need to know?**:
**Environment**:
- Dask version: 2.21.0, distributed version 2.30.0
- Python version: 3.7.8
- Operating System: MacOS Catalina 10.15.7
- Install method (conda, pip, source): pip
Contributor guide
Assessment
This issue has not been assessed yet.