open-telemetry / open-telemetry/opentelemetry-python-genai
Implement gen_ai.client.operation.time_to_first_chunk for OpenAI v2 streaming
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39
- Forks
- 63
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 175
Description
Problem
The OpenTelemetry GenAI Semantic Conventions define a gen_ai.client.operation.time_to_first_chunk histogram metric that records the time (in seconds) from request start to the first output chunk during streaming operations. No Python instrumentation currently implements this metric.
Originally tracked in opentelemetry-python-contrib #3932 with a previous implementation in opentelemetry-python-contrib#4415. That PR was closed when GenAI instrumentations moved to this repository.
Context
In the old repository, #4415 was designed to build on top of opentelemetry-python-contrib #4500 (streaming ABC refactor by @eternalcuriouslearner), which introduced the generic stream wrapper in opentelemetry-util-genai. That refactor is already present here at util/opentelemetry-util-genai/src/opentelemetry/util/genai/stream.py, so the time-to-first-chunk metric can now be built directly on top of it.
cc @lmolkova @eternalcuriouslearner
Proposed solution
- Add a
gen_ai.client.operation.time_to_first_chunkhistogram to the OpenAI v2 instrumentation and the generic streaming ABC. - Record
time_of_first_chunk - start_timein seconds when the first streaming chunk arrives. - Attach standard GenAI metric attributes:
gen_ai.operation.name,gen_ai.request.model,gen_ai.response.model,gen_ai.system,server.address,server.port. - Use the semconv-defined explicit bucket boundaries.
- Record only for streaming calls; do not emit when no chunk is ever received.
- Cover both sync and async streaming paths.
Acceptance criteria
- Metric recorded only on streaming completions (not non-streaming)
- No data point emitted if the stream errors before the first chunk
- Sync and async streaming paths both instrumented
- Test coverage ported from opentelemetry-python-contrib #4415
References
- Semantic convention: https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-metrics/
- Original issue: https://github.com/open-telemetry/opentelemetry-python-contrib/issues/3932
- Previous implementation: https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4415
- Streaming ABC refactor: https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4500
I have a previous implementation from opentelemetry-python-contrib#4415 ready to port.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with util/opentelemetry-util-genai/src/opentelemetry/util/genai/stream.py and the OpenAI v2 instrumentation, then review the implementation and tests from opentelemetry-python-contrib#4415. Trace both sync and async streaming paths and verify the histogram records only after a first chunk, with the required attributes and no point for errors or empty streams.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100