elastic / elastic/elastic-otel-python

Add universal profiling integration

Open
#32 0 comments 0 reactions 0 assignees View on GitHub
profiler
Dominant language
Python
Stars
15
Forks
18
Avg merge
1d 8h
Merged PRs (30d)
15

Description

We need to implement integration for universal profiling in order to enable trace correlation. Differently than the java agent I'd like to implement most of the code in Python and only pass the data to the C extension to store it in the well known variables.
After playing a bit with implementing a python extension I think the issue is more on sorting out the life cycle of things more than in the coding itself. For the process wide data this is mostly static so just have to do it at startup, we need to add a thread (ThreadManager?) that handles the communication with the profiling agent via the shared socket. For the threading specific one need to sort out where to update the data, maybe there's some hook in the context handling?
Another requirements is to buffer the spans for `samples-delay-ms` as sent in the profiler registration message, in order to let the profiler send the corresponding traces ids via the messaging socket. That would require a way for the thread reading the socket to send data to its parent process (use a Queue to send data and then implement of a cache that the exporter can use?).

Open issues from the profiler side:
- [ ] profiler registration message still not implemented
- [ ] renaming after OTel acceptance if any

Open issues with upstream OTel
- [ ] Context change hook https://github.com/open-telemetry/opentelemetry-python/issues/3915

Open issues from our side
- [ ] handle renaming if any
- [ ] sort out build and distributions with the robots team
- [ ] agree on the shared object built from the C extension file name
- [ ] handling of socket from thread
- [ ] when to update thread specific data
- [ ] how to pass data from thread reading socket
- [ ] hooks into batch span exporter to buffer spans for `samples-delay-ms`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.