dask / dask/distributed

Advertise active spans on Prometheus

Open
#7,976 4 comments 0 reactions 0 assignees View on GitHub
feature needs attention
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

- Follow-up to #7860

We would like to be able to link Prometheus metrics to spans.
At any given moment, the scheduler knows which spans are currently active - both in terms of tags and span ids - and could advertise such information to Prometheus.

For example, it would be great to be able to query Prometheus as follows:

> give me all the data from recent clusters while span tag "my preprocessing step" was executing somewhere on the cluster

Alternatively, from the client, you could take note of the span_id (yielded by the `span` context manager) and auto-generate a link to Grafana or another aggregator that shows the metrics while that specific span was active.

## Timeline
There is no timeline to deliver this feature at the moment of writing. I'm opening this ticket to keep track of the possible feature in the medium-long term.

CC @ntabris

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.