Azure / Azure/azure-sdk-for-python

[azure-ai-projects] ResponsesInstrumentor breaks openai with_raw_response streaming (AsyncStreamWrapper wraps an unparsed LegacyAPIResponse)

Aperta
#48,646 4 commenti 1 reazione 3 assegnatari Rivendicata da @M-Hietala Vedi su GitHub
AI Projects Client customer-reported needs-team-attention question Service Attention
Lingua principale
Python
Stelle
5.6k
Fork
3.4k
Merge medio
2g 2h
PR unite (30g)
213

Descrizione

### Describe the bug

With the Responses instrumentor active, `client.responses.with_raw_response.create(..., stream=True)` no longer returns a raw response. It returns `AsyncStreamWrapper`, which:

- does not expose `.parse()` or `.headers`, both part of the `with_raw_response` contract; and
- cannot be iterated, because its `stream_async_iter` is the still-unparsed `LegacyAPIResponse`, which has no `__anext__`.

So any caller that uses `with_raw_response` together with streaming breaks as soon as tracing is enabled, with:

```
AttributeError: 'LegacyAPIResponse' object has no attribute '__anext__'
```

`with_raw_response` is the only way to read response headers (e.g. `x-ms-served-model`) while streaming, so this affects callers that need per-response header data.

### Root cause

The instrumentor patches `openai.resources.responses.AsyncResponses.create`. `with_raw_response.create()` delegates to that same method, but sets the SDK's raw-response flag, so `create` returns a `LegacyAPIResponse` rather than an `AsyncStream`.

`_wrap_async_streaming_response` treats whatever `create` returned as the event stream and stores it as `stream_async_iter`, and `AsyncStreamWrapper.__anext__` calls `self.stream_async_iter.__anext__()`. For the raw-response path that object is a `LegacyAPIResponse`, which is not an async iterator — it must have `.parse()` called on it first.

Verified with a debug hook on `_wrap_async_streaming_response`:

```
[wrap] _wrap_async_streaming_response receives stream=openai._legacy_response.LegacyAPIResponse
```

### To Reproduce

Standalone: `openai` + `azure-ai-projects` only, no Azure resources needed (the transport is mocked, so the object graph is wrong before any real network call matters).

```python
import asyncio
import httpx
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from azure.core.settings import settings
from azure.core.tracing.ext.opentelemetry_span import OpenTelemetrySpan
from azure.ai.projects.telemetry._responses_instrumentor import ResponsesInstrumentor
from openai import AsyncOpenAI

trace.set_tracer_provider(TracerProvider())
settings.tracing_implementation = OpenTelemetrySpan

SSE = (
'data: {"type":"response.output_text.delta","delta":"Hello","item_id":"m1",'
'"output_index":0,"content_index":0,"sequence_number":1,"logprobs":[]}\n\n'
"data: [DONE]\n\n"
)
transport = httpx.MockTransport(
lambda r: httpx.Response(200, headers={"content-type": "text/event-stream"}, content=SSE.encode())
)

async def main() -> None:
ResponsesInstrumentor().instrument()
client = AsyncOpenAI(api_key="k", http_client=httpx.AsyncClient(transport=transport))

raw = await client.responses.with_raw_response.create(model="gpt-4o", input="hi", stream=True)

print(f"returned type : {type(raw).__module__}.{type(raw).__name__}")
print(f" .parse() : {hasattr(raw, 'parse')}")
print(f" .headers : {hasattr(raw, 'headers')}")
inner = getattr(raw, "stream_async_iter", None)
print(f" .stream_async_iter: {type(inner).__module__}.{type(inner).__name__}")
print(f" __anext__ : {hasattr(inner, '__anext__')}")

async for _ in raw:
pass

asyncio.run(main())
```

Note both an OpenTelemetry `TracerProvider` **and** `azure-core-tracing-opentelemetry` are required — without them `_create_responses_span_from_parameters` returns `None`, the instrumentor returns the result unwrapped, and the bug does not appear. That makes it easy to miss when reproducing.

### Expected behavior

`with_raw_response.create(..., stream=True)` should keep behaving like a raw response when tracing is on: `.parse()` and `.headers` available, and the parsed stream iterable — ideally with the wrapper still in the iteration path so telemetry is still recorded.

### Actual behavior

```
returned type : azure.ai.projects.telemetry._responses_instrumentor.AsyncStreamWrapper
.parse() : False <- raw-response API expects True
.headers : False <- raw-response API expects True
.stream_async_iter: openai._legacy_response.LegacyAPIResponse
__anext__ : False <- wrapper iterates this

iterating the returned object:
AttributeError: 'LegacyAPIResponse' object has no attribute '__anext__'
```

With the instrumentor uninstrumented, the same call correctly returns:

```
openai._legacy_response.LegacyAPIResponse .parse()=True .headers=True
```

### Environment

- `azure-ai-projects==2.3.0`
- `openai==2.53.0`
- `azure-core-tracing-opentelemetry==1.0.0b13`, `opentelemetry-sdk==1.44.0`
- Python 3.13, macOS (not platform specific)

### Notes

Reported downstream as microsoft/agent-framework#7461, where enabling `AZURE_EXPERIMENTAL_ENABLE_GENAI_TRACING=true` made all streaming agent requests fail. microsoft/agent-framework#7705 works around it by parsing the wrapper's inner raw response and handing it back, so the wrapper stays in the iteration path and telemetry is preserved. That workaround could be dropped once this is fixed upstream.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.