dask / dask/distributed

WARNING - Memory use is high but worker has no data to store to disk

Open
#4,193 17 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

I end up getting a lot of error messages like this:
```
distributed.worker - WARNING - Memory use is high but worker has no data to store to disk. Perhaps some other process is leaking memory? Process memory: 6.15 GB -- Worker memory limit: 8.45 GB
```
I'm relatively sure that this warning is actually true.
Also, the workers hitting this warning end up in idling all the time.

Is there a way to automatically retire + restart the workers that hit this error message?

**Anything else we need to know?**:
I'm using Dask v2.30

**Environment**:

- Dask version: v2.30
- Python version: 3.7.8
- Operating System: Centos 7
- Install method (conda, pip, source): conda

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.