grafana / grafana/pyroscope

Support jdk.OldObjectSample and Native Image JFR Memory Events in Pyroscope

Open
#4,752 0 comments 0 reactions 1 assignee Claimed by @korniltsev-grafanista View on GitHub
Dominant language
Go
Stars
11.7k
Forks
802
Avg merge
1d 19h
Merged PRs (30d)
80

Description

---

## Summary

Grafana Pyroscope currently does not parse or visualize memory-related Java Flight Recorder (JFR) events emitted by **GraalVM Native Images**, specifically the `jdk.OldObjectSample` event. Although these events are correctly captured in JFR recordings, they are ignored during ingestion, preventing heap-in-use or live-object profiling in Pyroscope dashboards.

---

## Environment

* **Runtime:** GraalVM Native Image
* **Profiling Source:** Java Flight Recorder (JFR)
* **Event Type:** `jdk.OldObjectSample`
* **Ingestion Pipeline:** Native Image → JFR → Grafana Alloy → Pyroscope
* **Pyroscope Deployment:** Self-hosted

---

## Observed Behavior

* Native image JFR successfully records memory-related events (verified using `jfr print`).
* `jdk.OldObjectSample` events are present in the JFR file.
* Pyroscope does **not** generate:

* Heap-in-use profiles
* Live object or retained object flame graphs
* No ingestion or parsing errors are reported.
* Memory-related JFR events appear to be ignored by the current `jfr-parser`.

---

## Expected Behavior

* Pyroscope should parse `jdk.OldObjectSample` events from JFR recordings.
* Heap-in-use or live-object profiles should be generated.
* Dashboards should allow visualization of long-lived objects to aid memory leak analysis in native images.

---

## Root Cause Analysis

* Pyroscope’s JFR ingestion currently supports a limited subset of events, primarily those produced by **async-profiler**.
* `jdk.OldObjectSample` is not supported by the current `jfr-parser`.
* Async-profiler cannot be used in GraalVM native images due to missing HotSpot APIs such as `AsyncGetCallTrace`.

---

## Why This Is Important

* GraalVM native images cannot use async-profiler.
* JFR is the only supported and safe profiling mechanism for native images.
* Without this support:
* Heap-in-use profiling is not possible in Pyroscope.
* Teams must rely on offline heap dumps, losing continuous observability.
* Supporting `jdk.OldObjectSample` would enable production-grade memory observability for native image workloads.

---

## Workarounds

* Periodic heap dumps can be used for memory leak analysis, but this approach is offline and not suitable for continuous profiling.
* There is no viable workaround for real-time heap-in-use visualization.

---

## References

* GraalVM Native Image JFR documentation:
[https://www.graalvm.org/latest/reference-manual/native-image/debugging-and-diagnostics/JFR/](https://www.graalvm.org/latest/reference-manual/native-image/debugging-and-diagnostics/JFR/)

---

## Additional Context

This behavior has been validated in a production-like environment.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.