grafana / grafana/pyroscope

Feature: Allow to customise the source code integration

Open
#4,339 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Go
Stars
11.7k
Forks
802
Avg merge
1d 19h
Merged PRs (30d)
80

Description

#### Is your feature request related to a problem? Please describe.

Our current implementation of source integration only support Go source code that is directly in the git repo or can be retrieved using go.mod mechnisms.

There is a huge matrix how source code gets assembled into or installed on the destination system, it might become very tricky to fully support all to modes for different languages/operating-system/build tools.

#### Describe the solution you'd like

It should be possible to configure a separate well defined external API, that would take care of all the intricacies and:

- Expose the right version of source code for a particular binary,
- The file path is exactly the same as it is seen on the profiling data

#### Proposal A: Allow to configure a custom URL for vcs.v1.GetFile / vcs.v1.GetCommit(s)

- This is the API we use today for retrieving file information, see [proto-definition]
- It works by passing `service_repository`, `service_git_ref`, `[service_root_path]` from labels on the profile.

[proto-definition]: https://github.com/grafana/pyroscope/blob/f9c4f2cfdd083d0a7f8f79ab5b289835fba73fd6/api/vcs/v1/vcs.proto#L9-L11

##### Example usage

```
# Get commit hash
curl 'http://service/vcs.v1.VCSService/GetCommit' \
-H 'content-type: application/json' \
--data-raw '{"repositoryURL":"https://github.com/grafana/pyroscope","ref":"HEAD"}'

# Get source code file
curl 'http://service/vcs.v1.VCSService/GetFile' \
-H 'content-type: application/json' \
--data-raw '{"repositoryURL":"https://github.com/grafana/pyroscope","ref":"HEAD","localPath":"/home/user/pyroscope/cmd/pyroscope/main.go","rootPath":""}'
{
"URL": "[URL to GitHub view]",
"content": "[returns the full source file as base64]"
}
```

#### Proposal B: Use debuginfod API for fetching source code

* This is well defined API, used by tools like gdb to fetch source and debuginfo to symbolize binaries.
* More on this https://wiki.archlinux.org/title/Debuginfod

We would need to make sure all profiles are propagating their build IDs in the [mapping object]. (e.g. golang doesn't do that in all cases). Theoretically we could also encaspulate the current information into a string and deliver through build IDs (`service_repository`, `service_git_ref`, `[service_root_path]`).

[mapping object]:https://github.com/google/pprof/blob/6e76a2b096b5fa52e4bb3f7f7a357bd6e6b3b7b1/proto/profile.proto#L169-L172

The server needs to be able to match the source code only by having the build ID, which is trickier that when the commit hash source code is known, as the build ID is only known at build time (e.g. in CI).

##### Example usage

```
# Get build id of python
$ file /usr/bin/python3.13
/usr/bin/python3.13: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=387f543579fb53103028f77de068e1e025ec148d, for GNU/Linux 3.7.0, stripped

# Request source code using path and build ID
$ curl https://debuginfod.fedoraproject.org/buildid/387f543579fb53103028f77de068e1e025ec148d/source/usr/src/debug/python3.13-3.13.5-1.fc42.aarch64/Programs/python.c
[returns the full file]
```

#### Additional context

Both of those API should be configured in the Grafana instance (with support for using basic and bearer token auth)

It might make sense to query multiple services per source code request, to support different source code retrieval forms for different languages/areas of the business.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.