grafana / grafana/pyroscope

Pyroscope v2 unable to elect itself as leader

Open
#5,257 2 comments 3 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
11.7k
Forks
802
Avg merge
1d 19h
Merged PRs (30d)
80

Description

#### Describe the bug
When running v2.0.3 as a single binary, pyroscope never becomes healthy and outputs the following log line repeatedly

```
ts=2026-06-13T14:08:32.318963752Z caller=compaction_service.go:96 level=error component=metastore msg="failed to prepare compaction plan" err="rpc error: code = Unavailable desc = node is not the leader"
```

This is after upgrading from v2.0.2 so I'm unsure if it's in the new version on a fresh instance or only after the upgrade, but it's very consistent for my environment.

#### To Reproduce
Steps to reproduce the behavior:

1. Run pyroscope helm chart v2.0.3 with the values file below

```yaml
pyroscope:
replicaCount: 1
cluster_domain: .cluster.local.

image:
repository: grafana/pyroscope
pullPolicy: IfNotPresent

extraArgs:
log.level: debug

dnsPolicy: ClusterFirst

rbac:
create: true

serviceAccount:
create: true

podAnnotations:
profiles.grafana.com/memory.scrape: "true"
profiles.grafana.com/memory.port_name: http2
profiles.grafana.com/cpu.scrape: "true"
profiles.grafana.com/cpu.port_name: http2
profiles.grafana.com/goroutine.scrape: "true"
profiles.grafana.com/goroutine.port_name: http2

podSecurityContext:
fsGroup: 10001
runAsUser: 10001
runAsNonRoot: true

resources:
limits:
memory: 1Gi
requests:
cpu: 200m
memory: 512Mi

persistence:
enabled: false
accessModes:
- ReadWriteOnce
size: 10Gi

alloy:
enabled: false

agent:
enabled: false

minio:
enabled: false

ingress:
enabled: false
```

#### Expected behavior

Pyroscope to be able to start and become ready

#### Environment

- Infrastructure: Kubernetes v1.33.6
- Deployment tool: Helm chart version 2.0.3 deployed through ArgoCD

#### Additional Context

Reverting to v2.0.2 resolves the problem for now and pyroscope is able to start without issue using the same values file

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.