grafana / grafana/pyroscope

Pyroscope garbles application logs

Open
#4,042 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
11.7k
Forks
802
Avg merge
1d 19h
Merged PRs (30d)
80

Description

#### Describe the bug

We have some log entries like this:

```
2025-03-22 20:23:30.174{"level":"INFO","ts":"1742675010174","deviceID":"1234","message":"Bla bla"}
[ERROR] Error uploading snapshot: 422 {"code":"unknown","message":"pushing IngestInput-pprof failed unknown: at least 2 live replicas required, could only find 1"}
```

This should be two log entries like this:

```
{"level":"INFO","ts":"1742675010174","deviceID":"1234","message":"Bla bla"}
2025-03-22 20:23:30.174 [ERROR] Error uploading snapshot: 422 {"code":"unknown","message":"pushing IngestInput-pprof failed unknown: at least 2 live replicas required, could only find 1"}
```

As you can see they are being interleaved. The second log line comes from pyroscope, the former from our application.

Unfortunately I'm not fully familiar with our infrastructure setup, especially pyroscope & grafana, and it would take me a few weeks of archeology to figure out answers to basic questions such as "how do we deploy pyroscope", etc.

All I really know is that the application is a JVM process, and we aggregate logs into Loki.

So I was just hoping that someone might be able to point me quickly to some documentation or a known issue where pyroscope is interleaving it's log lines with the JVM application. Does this problem have a solution? Can pyroscope be told to write it's logs to a different output? If so how?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.