grafana / grafana/pyroscope

Trace exemplars

Open
#3,289 2 comments 3 reactions 0 assignees View on GitHub
area/ui enhancement type/docs ux
Dominant language
Go
Stars
11.7k
Forks
802
Avg merge
1d 19h
Merged PRs (30d)
80

Description

We already have an integration with distributed tracing solutions that enables [traces to profiles](https://github.com/grafana/pyroscope/blob/main/examples/tracing/tempo/README.md) navigation. We should extend the scenario to enable the other way around: profiles to traces.

From the user perspective this could look like trace exemplars in metrics:
![image](https://github.com/grafana/pyroscope/assets/12090599/0948b9b7-d394-473e-9a61-b71a7833354f)

Given a query expression (series labels + call site filter), we could retrieve top K samples with tracing attribution (by count/sum) and place them in the timeline. The links would include `trace_id:span_id` pairs, which allows to navigate users straight to the trace span view. In turn, in the trace span view, there would show up a profile for this specific span.

![image](https://github.com/grafana/pyroscope/assets/12090599/26ffe7c7-9da0-4885-90e0-5bde922b8ef9)

This will require a few adjustments of the storage schema and SDKs (we only collect root span_id and span_name as of now).

It's possible that the trace referred by a profile was not sampled by the trace collector (tail sampling). This should be addressed in the collector

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.