asyncio exception when monitoring distributed compute
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
**Describe the issue**:
In the example code below, I try to monitor the progress of a cluster compute.
It does return the right results, but there is always an exception as shown below. This exception cannot be suppressed by try/except wrapper I add.
```
(1, 1, 1)
2023-11-30 21:27:09,396 - distributed.scheduler - ERROR -
Traceback (most recent call last):
File "/miniconda3/envs/X/lib/python3.9/site-packages/distributed/utils.py", line 832, in wrapper
return await func(*args, **kwargs)
File "/miniconda3/envs/X/lib/python3.9/site-packages/distributed/scheduler.py", line 7246, in feed
await asyncio.sleep(interval)
File "/miniconda3/envs/X/lib/python3.9/asyncio/tasks.py", line 652, in sleep
return await future
asyncio.exceptions.CancelledError
```
**Minimal Complete Verifiable Example**:
```python
from time import sleep
import dask
from dask.distributed import Client,progress
from dask_jobqueue import SGECluster
from dask import compute, persist, delayed
@dask.delayed
def f():
sleep(1)
return 1
cluster = SGECluster(cores=1, memory='1GB')
cluster.scale(1)
c = Client(cluster)
out=[ f() ]*3
try:
x=persist(*out)
progress(x)
out=compute(x)
except:
print("error")
c.close()
print(out[0])
```
**Anything else we need to know?**:
**Environment**:
- Dask version: 2023.11.0
- Python version: 3.9.18
- Operating System: Red Hat Enterprise Linux release 8.5 (Ootpa)
- Install method (conda, pip, source):
mamba install -c conda-forge dask
mamba install -c conda-forge dask-jobqueue
Contributor guide
Assessment
This issue has not been assessed yet.