dask / dask/distributed

Add hierarchy to published dashboard plots

Open
#5,130 7 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

In https://github.com/dask/dask-labextension/issues/179#issuecomment-882074574 we propose changing the Dask JupyterLab extension to provide a hierarchy of plots, for example "Worker Metrics", "Computation Progress", or "Profiling". Before the labextension could add something like this, the Dask scheduler probably needs to provide this hierarchy.

Currently the labextension queries for the individual-plots.json route, which is filled with the following code:

https://github.com/dask/distributed/blob/50fd3ff34e1a66e2fe0b27bce1457e8fd4b00d7d/distributed/http/scheduler/info.py#L149-L179

We should maybe change this code (or provide another route in parallel) that provided a more hierarchical view, maybe like the following (using yaml here just for convenience)

```yaml
real-time-worker-metrics:
- {"network": "individual-network-workers.html"}
- {"disk": "individual-disk-workers.html"}
- ...
aggregate-worker-metrics
- ...
scheduler-metrics:
- ...
gpu:
- ...
```

(Or some other, much better grouping). I think that this requires both an understanding of how Dask gets used, as well as enough understanding of the labextension to understand what it needs. To me this sounds like a job for @jacobtomlinson if he has time.

I think that this information should originate from this repository rather than dask-labextension so that we don't need to coordinate across codebases as we add plots.

cc also @ian-r-rose and @bryevdv

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.