Compactor: pyroscope_compaction_size_bytes histogram buckets are too small
- Dominant language
- Go
- Stars
- 11.7k
- Forks
- 802
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 80
Description
## Description
The `pyroscope_compaction_size_bytes` histogram has bucket boundaries that are far too small to capture actual compacted block sizes, making the metric effectively useless.
## Current bucket definition
https://github.com/grafana/pyroscope/blob/main/pkg/compactor/bucket_compactor.go#L290-L294
```go
m.Size = prometheus.NewHistogramVec(prometheus.HistogramOpts{
Name: "pyroscope_compaction_size_bytes",
Help: "Final block size after compaction by level",
Buckets: prometheus.ExponentialBuckets(32, 1.5, 12),
}, []string{"level"})
```
This generates buckets: `32, 48, 72, 108, 162, 243, 364, 546, 820, 1230, 1845, 2768` — maxing out at **~2.7 KB**.
## Problem
Compacted blocks are typically in the **MB to GB** range. Every observation lands in the `+Inf` bucket, so `histogram_quantile()` returns a flat value near the last finite bucket boundary (~3KB) regardless of actual block sizes.
```promql
-- Always returns ~3KB regardless of actual compaction sizes
histogram_quantile(0.95, sum(rate(pyroscope_compaction_size_bytes_bucket{cluster="$cluster"}[15m])) by (le))
```
## Suggested fix
Use buckets that cover the realistic range of compacted block sizes (MB to GB):
```go
Buckets: prometheus.ExponentialBuckets(1<<20, 2, 15),
// 1MB, 2MB, 4MB, 8MB, 16MB, 32MB, 64MB, 128MB, 256MB, 512MB, 1GB, 2GB, 4GB, 8GB, 16GB
```
Alternatively, this could be made configurable so operators can tune the bucket boundaries to match their workload characteristics.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.