dask / dask/distributed

Support multiple measures in `MemorySampler`

Open
#6,864 4 comments 0 reactions 0 assignees View on GitHub
diagnostics enhancement
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

It would be handy to be able to record multiple measures at once in `MemorySampler`. Particularly, recording `process` and `managed_spilled` at the same time gives you a picture of both RAM and disk usage. But also tracking any of the metrics could be nice.

This is easy to do. The one small question is how to structure the DataFrame, which currently has a timeseries index, and one column per `sample` call. This effectively adds another dimension (`measure`) to the data, so using an xarray Dataset would actually be nicest here, though we could also just make a DataFrame in tidy format [like xarray does](https://docs.xarray.dev/en/stable/user-guide/pandas.html#hierarchical-and-tidy-data).

To be used in https://github.com/coiled/coiled-runtime/issues/191.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.