intel / intel/gprofiler

python: py-spy: run without --gil

Open
#501 0 comments 0 reactions 0 assignees View on GitHub
runtime/python
Dominant language
Python
Stars
826
Forks
79
PR merge metrics
No merged PRs in 30d

Description

Currently we're running pyspy in `--gil` mode: this means that we don't record threads that don't hold the GIL.

This may yield misleading results: for example, a function releasing the GIL but still consuming CPU time won't be recorded.
Our merging logic scales all samples taken by pyspy according to the number of samples that `perf record` has collected. So, if we have a Python process with 5 threads consuming CPU in parallel, only 1 will be recorded by pyspy (the one running Python) and when scaling we will hide all work done by the non-Python threads.

We should try improving that. The reason we started with `--gil` is that, as I recall, pyspy doesn't identify running threads very good (you can run `pyspy record ...` on a sleeping app and it will produce tons of samples...).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.