dask / dask/distributed

Improved layout for task graph in dashboard

Open
#3,467 3 comments 6 reactions 0 assignees View on GitHub
diagnostics
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

Over the weekend I played around with improving the visuals of the task graph in the online dashboard and was wondering if you were interested in me refining this for a PR.

Some examples:
* The [custom graph from the documentation ](https://docs.dask.org/en/latest/custom-graphs.html) becomes
![dask_graph_2](https://user-images.githubusercontent.com/25567300/74268539-4f13db80-4d08-11ea-9d04-c6390e6dc231.jpg)

* The code
```python
from distributed import Client, LocalCluster
import dask.array as da

cluster = LocalCluster()
client = Client(cluster)
x = da.ones((1500, 1500), chunks=(300, 300))

y = x + x.T

client.compute(y)
```
becomes
![dask_graph_1](https://user-images.githubusercontent.com/25567300/74268690-913d1d00-4d08-11ea-9072-e1f2c73e80a5.jpg)

* A complex custom graph from my project at work transforms from this
![dask_custom_graph_old](https://user-images.githubusercontent.com/25567300/74268784-be89cb00-4d08-11ea-8813-a885d2263829.png)
to this
![dask_custom_graph_new](https://user-images.githubusercontent.com/25567300/74268801-c77a9c80-4d08-11ea-98fd-730bdbd94175.png)

---

To get these results I did the following
* slightly tuned the colors to [more eye friendly versions ](http://colorbrewer2.org/?type=qualitative&scheme=Set1&n=3) (I just noted that #2711 exists) - loosely following [this ](https://commons.wikimedia.org/wiki/File:Topological_Ordering.svg#/media/File:Topological_Ordering.svg) wikipedia visualization
* replaced the straight edges with bezier curves
* added arrow heads to the edges
* removed the axes as they don't really add anything to the visualization
* replaced the `update_graph()` method in `graph_layout.py` with a version leveraging on the [Grandalf](https://github.com/bdcht/grandalf) package to compute a nicer layout
* my initial impression is that the overhead caused by proper layouting is not an issue for graphs not larger than few hundred nodes
* for a final implementation one could use the old method as an automatic fallback in case of larger graphs or allow selecting the method via a config entry
* dask already has graphviz as an optional dependency and it might make sense to use this instead to compute the node coordinates (though it's a heavier dependency, whereas Grandalf is self contained, pure Python)
* it might also make sense to switch to a pure client side solution (e.g. based on d3.js or better yet something canvas or webgl #2976 based)

Any thoughts?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.