traceloop / traceloop/openllmetry
๐ Bug Report: Groq streaming calls record no token or duration metrics
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.4k
- Forks
- 1.1k
- Avg merge
- 8d 14h
- Merged PRs (30d)
- 2
Description
Which component is this bug for?
Groq Instrumentation
๐ Description
Streaming calls record no metrics at all. Both request handlers in opentelemetry/instrumentation/groq/__init__.py return into the stream processor before the metric block is reached:
if is_streaming_response(response):
return _create_stream_processor(response, span, event_logger)
elif response:
metric_attributes = shared_metrics_attributes(response)
if duration_histogram:
duration_histogram.record(duration, attributes=metric_attributes)
_handle_response(span, response, token_histogram, event_logger)
_create_stream_processor and _create_async_stream_processor do not take token_histogram or duration_histogram, so nothing in the streaming path can record. gen_ai.client.token.usage and gen_ai.client.operation.duration are emitted for non-streaming calls only.
The span is unaffected. set_model_streaming_response_attributes still sets the token attributes on it, so the trace looks correct and nothing logs a warning. The gap only shows up as missing series on a dashboard.
The usage data needed is already collected. _process_streaming_chunk pulls it off chunk.x_groq.usage and the accumulated value reaches _handle_streaming_response, it is just never recorded to the histogram.
๐ Reproduction steps
Instrument with a metric reader and make one streaming call:
reader = InMemoryMetricReader()
GroqInstrumentor().instrument(meter_provider=MeterProvider(metric_readers=[reader]))
response = client.chat.completions.create(
model="llama-3.3-70b-versatile",
messages=[{"role": "user", "content": "hi"}],
stream=True,
)
for _ in response:
pass
print(reader.get_metrics_data())
๐ Expected behavior
A streaming call records the same token usage and duration metrics as the equivalent non-streaming call.
๐ Actual Behavior with Screenshots
get_metrics_data() returns nothing. The same call without stream=True records both gen_ai.client.token.usage and gen_ai.client.operation.duration.
๐ค Python Version
3.12
๐ Provide any additional context for the Bug.
The package has no tests/metrics/ directory, which is why this was not caught. The openai package has one, sharing fixtures from a tests/conftest.py one level above tests/traces/. Groq keeps its conftest inside tests/traces/, so adding metric tests means moving that file up.
๐ Have you spent some time to check if this bug has been raised before?
- I checked and didn't find similar issue
๐ข Have you read the Contributing Guidelines?
- I have read the Contributing Guidelines
Are you willing to submit PR?
Yes I am willing to submit a PR!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up โ it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in opentelemetry/instrumentation/groq/init.py and trace _create_stream_processor, _create_async_stream_processor, _process_streaming_chunk, and _handle_streaming_response. Use the provided InMemoryMetricReader reproduction, then review the Groq tests/traces/conftest.py and the OpenAI metrics tests for test structure. Done means streaming calls expose token usage and duration metrics like non-streaming calls, with regression coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100