traceloop / traceloop/openllmetry

๐Ÿ› Bug Report: Groq streaming calls record no token or duration metrics

Open
#4,419 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.4k
Forks
1.1k
Avg merge
8d 14h
Merged PRs (30d)
2

Description

Which component is this bug for?

Groq Instrumentation

๐Ÿ“œ Description

Streaming calls record no metrics at all. Both request handlers in opentelemetry/instrumentation/groq/__init__.py return into the stream processor before the metric block is reached:

if is_streaming_response(response):
    return _create_stream_processor(response, span, event_logger)
elif response:
    metric_attributes = shared_metrics_attributes(response)
    if duration_histogram:
        duration_histogram.record(duration, attributes=metric_attributes)
    _handle_response(span, response, token_histogram, event_logger)

_create_stream_processor and _create_async_stream_processor do not take token_histogram or duration_histogram, so nothing in the streaming path can record. gen_ai.client.token.usage and gen_ai.client.operation.duration are emitted for non-streaming calls only.

The span is unaffected. set_model_streaming_response_attributes still sets the token attributes on it, so the trace looks correct and nothing logs a warning. The gap only shows up as missing series on a dashboard.

The usage data needed is already collected. _process_streaming_chunk pulls it off chunk.x_groq.usage and the accumulated value reaches _handle_streaming_response, it is just never recorded to the histogram.

๐Ÿ‘Ÿ Reproduction steps

Instrument with a metric reader and make one streaming call:

reader = InMemoryMetricReader()
GroqInstrumentor().instrument(meter_provider=MeterProvider(metric_readers=[reader]))

response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "hi"}],
    stream=True,
)
for _ in response:
    pass

print(reader.get_metrics_data())
๐Ÿ‘ Expected behavior

A streaming call records the same token usage and duration metrics as the equivalent non-streaming call.

๐Ÿ‘Ž Actual Behavior with Screenshots

get_metrics_data() returns nothing. The same call without stream=True records both gen_ai.client.token.usage and gen_ai.client.operation.duration.

๐Ÿค– Python Version

3.12

๐Ÿ“ƒ Provide any additional context for the Bug.

The package has no tests/metrics/ directory, which is why this was not caught. The openai package has one, sharing fixtures from a tests/conftest.py one level above tests/traces/. Groq keeps its conftest inside tests/traces/, so adding metric tests means moving that file up.

๐Ÿ‘€ Have you spent some time to check if this bug has been raised before?
  • I checked and didn't find similar issue
๐Ÿข Have you read the Contributing Guidelines?
  • I have read the Contributing Guidelines
Are you willing to submit PR?

Yes I am willing to submit a PR!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up โ€” it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in opentelemetry/instrumentation/groq/init.py and trace _create_stream_processor, _create_async_stream_processor, _process_streaming_chunk, and _handle_streaming_response. Use the provided InMemoryMetricReader reproduction, then review the Groq tests/traces/conftest.py and the OpenAI metrics tests for test structure. Done means streaming calls expose token usage and duration metrics like non-streaming calls, with regression coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.