traceloop / traceloop/openllmetry

๐Ÿ› Bug Report: streaming choice events use the drained generator instead of the last chunk

Open Beginner friendly
#4,417 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.4k
Forks
1.1k
Avg merge
8d 14h
Merged PRs (30d)
2

Description

Which component is this bug for?

Google Generative AI Instrumentation

๐Ÿ“œ Description

Both streaming builders in opentelemetry/instrumentation/google_generativeai/__init__.py pass the generator to emit_choice_events instead of the chunk:

complete_response = "".join(text_parts)

if emit_events:
    emit_choice_events(response, event_logger)
else:
    if last_chunk is not None and getattr(last_chunk, "candidates", None):
        set_response_attributes(span, last_chunk, llm_model)

By that point response has been fully drained by the for item in response loop above. emit_choice_events does for index, candidate in enumerate(response.candidates), and a generator has no candidates. The non-events branch immediately below already uses last_chunk, which is the object that carries them.

This only fires when events are enabled, so use_legacy_attributes=False.

Two things make it worse than a dropped span. _build_from_streaming_response is not decorated @dont_throw, unlike the other handlers in the file, and it has no try/finally around span.end(). So the AttributeError is raised out of the generator on the final next(), into the calling application, and the span is never ended or exported.

๐Ÿ‘Ÿ Reproduction steps

Instrument with events enabled and stream a response:

GoogleGenerativeAiInstrumentor(use_legacy_attributes=False).instrument()

for chunk in client.models.generate_content_stream(model="gemini-2.0-flash", contents="hi"):
    print(chunk.text)
๐Ÿ‘ Expected behavior

Choice events are emitted from the final chunk, the stream is consumed cleanly, and the span is exported.

๐Ÿ‘Ž Actual Behavior with Screenshots
AttributeError: 'generator' object has no attribute 'candidates'

Raised into the caller on the last iteration. Spans exported: 0. With use_legacy_attributes=True the same call streams cleanly and exports one span.

๐Ÿค– Python Version

3.12

๐Ÿ“ƒ Provide any additional context for the Bug.

An empty stream needs a guard too, since last_chunk stays None there.

Separately, the missing @dont_throw and span.end() in a finally on these builders is worth addressing, but I have left that out to keep the change focused.

๐Ÿ‘€ Have you spent some time to check if this bug has been raised before?
  • I checked and didn't find similar issue
๐Ÿข Have you read the Contributing Guidelines?
  • I have read the Contributing Guidelines
Are you willing to submit PR?

Yes I am willing to submit a PR!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up โ€” it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in opentelemetry/instrumentation/google_generativeai/init.py and inspect both streaming builders, the response-draining loop, and emit_choice_events. Reproduce the issue with GoogleGenerativeAiInstrumentor(use_legacy_attributes=False) and generate_content_stream; done means choice events use the final chunk, empty streams are guarded, and the stream completes with its span exported.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.