grafana / grafana/pyroscope

The sample count displayed is wrong

Open
#3,340 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Go
Stars
11.7k
Forks
802
Avg merge
1d 19h
Merged PRs (30d)
80

Description

#### Describe the bug

Our currently displayed sample count in the UI is wrong. It seems to represent the sum of values multiplied by 10e9. In order to correct this we need to signal the sampling rate from the collection point (SDK/alloy/agent) and persist it in Pyroscope.

The OG Pyroscope endpoint has this metadata, which is unused by the adapter:

https://github.com/grafana/pyroscope/blob/ff0426278f16120b951cb4466b103349b0097c9d/pkg/og/ingestion/ingestion.go#L50

We then need to calculate the average sample rate for the profiles in a flamegraph (sample rate is a flamegraph wide parameter in flamebearer) and update our read path accordingly.

#### To Reproduce
Steps to reproduce:

1. Query e.g. a CPU profile:

image

Both the value in the tool tip and and the total value, are wrong and just derived by

#### Expected behavior

This is the hard part to define the right count of samples in a flameGraph.

For time unit profiles:

* Sum of values / sampleRate, while the sampleRate is only valid per profile

For memory/object unit profiles:

* Unsure what we can do here, go for example scales up the sampled in-use space to represent the actual memory usage

#### Next steps

Variant A:

- [ ] Update the default value for CPU to be more correct for defaults:

Variant B:

- [ ] Persist sample rate per profile
- [ ] Adapt query path to use avg. sampleRate for profiles in a flamegraph
- [ ] Adapt query path display indivual sample counts per flamegraph box

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.