traceloop / traceloop/openllmetry

Responses API streaming crashes with 'AsyncStream' object has no attribute 'id' when using .with_raw_response

Open
#4,476 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.4k
Forks
1.1k
Avg merge
8d 14h
Merged PRs (30d)
2

Description

Which component is this bug for?

OpenAI Instrumentation

📜 Description

client.responses.with_raw_response.create(stream=True, ...) crashes async_responses_get_or_create_wrapper with:

AttributeError: 'AsyncStream' object has no attribute 'id'

This is a variant of #4058 / PR #4078 that the fix didn't cover. #4078 taught parse_response/async_parse_response
to unwrap APIResponse/AsyncAPIResponse (used by .with_streaming_response), but .with_raw_response.create() on
Responses returns a LegacyAPIResponse (via openai._legacy_response.async_to_raw_response_wrapper), which was
already unwrapped before #4078. The problem isn't the unwrapping itself — it's that LegacyAPIResponse.parse(),
when the underlying request was stream=True, returns a Stream/AsyncStream instance, not a parsed Response.

The only isinstance(response, (Stream, AsyncStream)) check in async_responses_get_or_create_wrapper runs
before parsing, on the raw, unparsed return of wrapped():

https://github.com/traceloop/openllmetry/blob/main/packages/opentelemetry-instrumentation-openai/opentelemetry/instrumentation/openai/v1/responses_wrappers.py#L666

When the call goes through .with_raw_response, that raw return is a LegacyAPIResponse wrapper, not a
Stream/AsyncStream, so the check misses it. Execution falls through, async_parse_response() calls
.parse() on it, which (per the self._stream branch in openai._legacy_response.LegacyAPIResponse._parse)
returns an AsyncStream. There's no post-parse recheck for Stream/AsyncStream, so it crashes reading .id:

https://github.com/traceloop/openllmetry/blob/main/packages/opentelemetry-instrumentation-openai/opentelemetry/instrumentation/openai/v1/responses_wrappers.py#L746

This combination (.with_raw_response + stream=True) is exactly what agent_framework_openai (Microsoft Agent
Framework) uses as of >=1.6.0, to read the x-ms-served-model response header — so every agentic streaming turn
instrumented with this package crashes.

👟 Reproduction steps

Self-contained (mocks the HTTP transport, no API key needed):

import asyncio
import httpx
from opentelemetry.instrumentation.openai import OpenAIInstrumentor
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry import trace

trace.set_tracer_provider(TracerProvider())
OpenAIInstrumentor().instrument()

from openai import AsyncOpenAI  # noqa: E402

SSE_BODY = (
    b'event: response.created\n'
    b'data: {"type":"response.created","response":{"id":"resp_123","object":"response",'
    b'"created_at":0,"status":"in_progress","model":"gpt-4o-mini","output":[]}}\n\n'
    b'event: response.completed\n'
    b'data: {"type":"response.completed","response":{"id":"resp_123","object":"response",'
    b'"created_at":0,"status":"completed","model":"gpt-4o-mini","output":[],'
    b'"usage":{"input_tokens":1,"output_tokens":1,"total_tokens":2}}}\n\n'
)

def handler(request: httpx.Request) -> httpx.Response:
    return httpx.Response(
        200,
        headers={"content-type": "text/event-stream", "x-ms-served-model": "gpt-4o-mini"},
        content=SSE_BODY,
    )

async def main():
    client = AsyncOpenAI(api_key="sk-test", http_client=httpx.AsyncClient(transport=httpx.MockTransport(handler)))
    raw = await client.responses.with_raw_response.create(
        model="gpt-4o-mini",
        input="hello",
        stream=True,
    )
    stream = raw.parse()
    async for event in stream:
        pass
    print("no crash")

asyncio.run(main())
👍 Expected behavior

No crash — either trace the stream normally (matching the non-with_raw_response path) or skip tracing gracefully.

👎 Actual Behavior

Reproduced on both:

  • opentelemetry-instrumentation-openai==0.62.3 + openai==2.50.0
  • opentelemetry-instrumentation-openai==0.62.3 + openai==3.14.1 (latest)
File ".../opentelemetry/instrumentation/openai/v1/responses_wrappers.py", line 746, in async_responses_get_or_create_wrapper
    existing_data = responses.get(parsed_response.id)
                                  ^^^^^^^^^^^^^^^^^^
AttributeError: 'AsyncStream' object has no attribute 'id'
🤖 Python Version

3.12

📃 Additional context

We're currently working around this in our own codebase by unwrapping just the Responses API span instrumentation
after OpenAIInstrumentor().instrument() runs.

Are you willing to submit PR?

Yes I am willing to submit a PR if someone can confirm my assessment of the issue.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in opentelemetry/instrumentation/openai/v1/responses_wrappers.py, especially async_responses_get_or_create_wrapper around the cited lines, and run the self-contained AsyncOpenAI reproduction with with_raw_response and stream=True. Trace the raw and parsed response paths; done means the reproduction completes without the AsyncStream.id crash and either instruments the stream normally or skips it gracefully.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.