dask / dask/distributed

Set worker memory limit to current free memory, not total memory?

Open
#6,895 5 comments 1 reaction 0 assignees View on GitHub
memory
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

In https://github.com/coiled/feedback/issues/185, we found that system services were taking up enough memory that the worker setting its memory limit to to the total amount of virtual memory available often caused it to put the system under memory pressure and freeze it before the terminate limit could kick in.

Perhaps dask should take this approach in general, and rather than setting its default memory limit to the total system memory (which is always going to be too high), it should use the amount of currently-available memory (maybe plus its own current RSS)?

We'd just change `.total` here to `.available`:
https://github.com/dask/distributed/blob/969aa4636bff9a4c9e1f3805fdaf9cc4d44ab1db/distributed/system.py#L19

cc @fjetter @ntabris

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.