grafana / grafana/pyroscope-python

Not getting profiles of custom native cpp extensions of our libraries when using pyroscope

Open
#61 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
4
Forks
2
Avg merge
2d 4h
Merged PRs (30d)
5

Description

Hi,

Thanks for your great work!

We are using pyroscope to get profiles of our custom python code. From it, with pyroscope we are only being able to profile the python functions, but we get no detail of the c/c++ extensions.

For debugging this, we tried plain py-spy. With py-spy, we needed to:
1. Compile our custom extensions with the ` -fno-omit-frame-pointer` target compile option.
2. Adding `-Ccmake.build-type=Debug` to our pip install comand.
3. Activate the _native_ flag on `py-spy` for generating the flame graph:
``` python
py-spy record --native -o ./profile.svg -- python script_to_be_profiled.py
```

With these steps, our flamegraph recovered all the native extensions detail on its profiling.

However, with pyroscope, keeping the steps 1 and 2, we were not able to activate these options.

We saw that the `native` flag has a deprecation warning, and that on the `lib.rs` file, the config interface hardcodes the native parameter to false:

https://github.com/grafana/pyroscope-python/blob/dfe738d83605f1be8f99b47722fb87c9b19618ed/rust/src/lib.rs#L107-L119

Can pyroscope be used to profile the python native extensions? If so, what could we be missing in our setup?

Contributor guide

No contributing guide indexed for this repository

Research direction

Inspect rust/src/lib.rs at lines 107-119, where the native setting is hardcoded, and compare its configuration with the py-spy command using --native. Reproduce profiling with the stated frame-pointer and Debug build options. Done means determining whether native-extension profiling can be enabled and identifying the required configuration or implementation change.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python, rust
Domain
observability, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.