dask / dask/distributed

Log and display cluster events

Open
#921 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

We maintain structured logs about various fine grained events on each task in `Scheduler.transition_log`, `Worker.log`, `Stealing.log`, etc.. these are invaluable both in tracking down issues with methods like `Scheduler.transition_story(*keys)` and `Worker.story(*keys)`, and in producing visuals.

As people use clusters more dynamically we may find value in recording events around workers, clients, consistency checks, etc.. Ideally we find a consistent way to do this with many kinds of events that makes creating plots or query functions easy in the future.

So some questions:

1. What are good events to track?
2. What are good consistency checks to track?
3. How do we organize these events for easy querying and visualization?
4. What visuals and queries would be of pragmatic use?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.