Feature: Allow to customise the source code integration
- Dominant language
- Go
- Stars
- 11.7k
- Forks
- 802
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 80
Description
#### Is your feature request related to a problem? Please describe.
Our current implementation of source integration only support Go source code that is directly in the git repo or can be retrieved using go.mod mechnisms.
There is a huge matrix how source code gets assembled into or installed on the destination system, it might become very tricky to fully support all to modes for different languages/operating-system/build tools.
#### Describe the solution you'd like
It should be possible to configure a separate well defined external API, that would take care of all the intricacies and:
- Expose the right version of source code for a particular binary,
- The file path is exactly the same as it is seen on the profiling data
#### Proposal A: Allow to configure a custom URL for vcs.v1.GetFile / vcs.v1.GetCommit(s)
- This is the API we use today for retrieving file information, see [proto-definition]
- It works by passing `service_repository`, `service_git_ref`, `[service_root_path]` from labels on the profile.
[proto-definition]: https://github.com/grafana/pyroscope/blob/f9c4f2cfdd083d0a7f8f79ab5b289835fba73fd6/api/vcs/v1/vcs.proto#L9-L11
##### Example usage
```
# Get commit hash
curl 'http://service/vcs.v1.VCSService/GetCommit' \
-H 'content-type: application/json' \
--data-raw '{"repositoryURL":"https://github.com/grafana/pyroscope","ref":"HEAD"}'
# Get source code file
curl 'http://service/vcs.v1.VCSService/GetFile' \
-H 'content-type: application/json' \
--data-raw '{"repositoryURL":"https://github.com/grafana/pyroscope","ref":"HEAD","localPath":"/home/user/pyroscope/cmd/pyroscope/main.go","rootPath":""}'
{
"URL": "[URL to GitHub view]",
"content": "[returns the full source file as base64]"
}
```
#### Proposal B: Use debuginfod API for fetching source code
* This is well defined API, used by tools like gdb to fetch source and debuginfo to symbolize binaries.
* More on this https://wiki.archlinux.org/title/Debuginfod
We would need to make sure all profiles are propagating their build IDs in the [mapping object]. (e.g. golang doesn't do that in all cases). Theoretically we could also encaspulate the current information into a string and deliver through build IDs (`service_repository`, `service_git_ref`, `[service_root_path]`).
[mapping object]:https://github.com/google/pprof/blob/6e76a2b096b5fa52e4bb3f7f7a357bd6e6b3b7b1/proto/profile.proto#L169-L172
The server needs to be able to match the source code only by having the build ID, which is trickier that when the commit hash source code is known, as the build ID is only known at build time (e.g. in CI).
##### Example usage
```
# Get build id of python
$ file /usr/bin/python3.13
/usr/bin/python3.13: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=387f543579fb53103028f77de068e1e025ec148d, for GNU/Linux 3.7.0, stripped
# Request source code using path and build ID
$ curl https://debuginfod.fedoraproject.org/buildid/387f543579fb53103028f77de068e1e025ec148d/source/usr/src/debug/python3.13-3.13.5-1.fc42.aarch64/Programs/python.c
[returns the full file]
```
#### Additional context
Both of those API should be configured in the Grafana instance (with support for using basic and bearer token auth)
It might make sense to query multiple services per source code request, to support different source code retrieval forms for different languages/areas of the business.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.