Task Group visualization doesn't scale to thousands of task groups
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
I just saw an example where there were a thousand TaskGroups, each of size one. This is arguably the fault of the code that produced such a wonky setup, but we should be robust to it.
As a solution, I propose that if there are more than, say, 200 TaskGroups, that we only display those that have many tasks
Actually, that doesn't quite work for a few reasons:
1. We may lose important dependencies like `sum()`
2. That may break parts of the graph, resulting in odd holes, killing our layout algorithm
But! Maybe we can be clever here in some way! Thoughts welcome. cc @ncclementi @jrbourbeau
In the specific case that I saw the thousand single-task groups were all data creation tasks without dependencies. @ian-r-rose may be able to provide a reproducer.
Contributor guide
Assessment
This issue has not been assessed yet.