dask / dask/distributed

Improve configuration/composition of Prometheus metric collection

Open
#7,364 11 comments 0 reactions 0 assignees View on GitHub
diagnostics
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

As of now, our Prometheus collectors are hard-coded into their respective submodules. This makes it hard to make changes to the metrics we expose. In particular, I see two use cases where better configuration/composability would benefit users:

1. Some of the metrics we expose are very detailed and mostly useful for debugging and development purpose. Since there is a cost to each collected metric (in terms of storage, processing time, or $ spent on Cloud services), it would be helpful to be able to configure which (sets of) metrics we want to expose for scraping.
2. Extensions or plugins may want to expose additional metrics, see the `WorkStealingMetricCollector` as an example. For these, having an easy way to (un)register collectors would be great.

The first point seems to be related to the second one. If we had an easy way to (un)register collectors, we could have a variety of optional collectors that users could register if desired.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.