dask / dask/distributed

Diagnostics page for Adaptive decisions

Open
#3,647 0 comments 2 reactions 0 assignees View on GitHub
adaptive
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

It can be somewhat hard to determine when / why the scheduler decides to scale the cluster under adaptive mode. Ideally a dashboard page could shed some light here.

We currently have `/json/counts.json` which provides `desired_workers`. I think that's it.

I think there are two main pieces of information to convey:

1. Stock: The current state of things including current CPU load, current CPU capacity, and the current desired CPU capacity. Likewise for memory
2. Flow: The history of decisions on when to scale up / down the cluster (ideally with information on why those decisions were made (the state at that time)

Here's a rough sketch for number 1.

![Adaptive sketch](https://user-images.githubusercontent.com/1312546/77762443-895ee100-7007-11ea-8aba-77d878b9c775.png)

cc @rsignell-usgs, @jsignell for adaptive things, and @jacobtomlinson for dashboard design things.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.