dask / dask/distributed

Sort dashboard progress bars in topological order?

Open
#6,983 1 comment 2 reactions 0 assignees View on GitHub
diagnostics
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

I believe the progress bars on the dashboard are currently sorted by group size (largest first):

https://github.com/dask/distributed/blob/bfc5cfea80450954dba5b87a5858cb2e3bac1833/distributed/diagnostics/progress_stream.py#L94

This is a cheap metric that probably sometimes approximates topological order. Of course, it's wrong for any fan-out operations (repartition, shuffle, etc.).

But progress might be easier to watch and decipher if it was in actual topological order. Bars would then be most full at the top, and least full at the bottom.

It took me a long time of using dask to actually understand what the progress bars were showing, I think because it felt so random which ones were completing first.

It seems doable to maintain topological ordering, but might require some state in between updates to do efficiently.

### Current

![image](https://user-images.githubusercontent.com/3309802/187813955-ffec9623-0b17-4b6a-98d8-8ebfceb825a3.png)

_In the above example, `make-timeseries` comes first in topological order, then the `repartition`s, then `sub`, then `dataframe-count` and `dataframe-sum`_

### Proposed

![progress](https://user-images.githubusercontent.com/3309802/187815462-3d63172a-6120-4701-8ceb-c379ef05ef2e.png)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.