MemoryState is taking non trivial amount of CPU time in dashboard
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
I did some profiling on larger clusters (see also https://github.com/dask/distributed/pull/8725) and noticed the construction of [`MemoryState`](https://github.com/fjetter/distributed/blob/4c426a545e35c03acc6533b81a1ec8aa3ad7a664/distributed/scheduler.py#L262-L311) to take a non-trivial amount of time despite it being defined as using slots.
This shows up particularly in places where the full state object is not required, e.g.
- https://github.com/fjetter/distributed/blob/4c426a545e35c03acc6533b81a1ec8aa3ad7a664/distributed/dashboard/components/scheduler.py#L394
- https://github.com/fjetter/distributed/blob/4c426a545e35c03acc6533b81a1ec8aa3ad7a664/distributed/dashboard/components/scheduler.py#L4258
In https://github.com/fjetter/distributed/blob/4c426a545e35c03acc6533b81a1ec8aa3ad7a664/distributed/scheduler.py#L1849 we are creating possibly hundreds of those objects just to throw them away right away.
I think if we wanted to maintain such a MemoryState object we should ensure that the instances are being updated instead of recreating them on demand given that those dashboards update every 100ms.
Overall this is just 3% of CPU time in my measurement (about the same size as https://github.com/dask/distributed/pull/8725) so it is not urgent.
Contributor guide
Assessment
This issue has not been assessed yet.