Log and display cluster events
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
We maintain structured logs about various fine grained events on each task in `Scheduler.transition_log`, `Worker.log`, `Stealing.log`, etc.. these are invaluable both in tracking down issues with methods like `Scheduler.transition_story(*keys)` and `Worker.story(*keys)`, and in producing visuals.
As people use clusters more dynamically we may find value in recording events around workers, clients, consistency checks, etc.. Ideally we find a consistent way to do this with many kinds of events that makes creating plots or query functions easy in the future.
So some questions:
1. What are good events to track?
2. What are good consistency checks to track?
3. How do we organize these events for easy querying and visualization?
4. What visuals and queries would be of pragmatic use?
Contributor guide
Assessment
This issue has not been assessed yet.