Streaming-path parity for gpt-4o-audio: surface `delta.audio` (follow-up to #517)
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 157
- Forks
- 32
- Avg merge
- 1h 25m
- Merged PRs (30d)
- 145
Description
Context / follow-up to #517
#517 surfaced gpt-4o-audio's message.audio on the non-stream /v1/chat/completions response. The streaming path still drops audio:
OpenAiStreamDelta(crates/aisix-provider-openai/src/wire.rs) does not modelaudio.ChatDeltahas noextrapassthrough.RenderedDelta(crates/aisix-proxy/src/render.rs) has noaudio/extra.
OpenAI streams gpt-4o-audio output via choices[].delta.audio ({ id?, data?, transcript?, expires_at? } incremental chunks). With the streaming structs not modeling it, a customer streaming a gpt-4o-audio response gets no audio deltas.
This is the same non-stream-vs-stream split as reasoning (#466 non-stream → #502 streaming parity).
Ask
Surface delta.audio on the streaming path: model it on OpenAiStreamDelta, carry it through ChatDelta (likely via an extra flatten mirroring the message-level approach), and emit it on RenderedDelta so streamed gpt-4o-audio audio reaches the client.
Verification
- Mock-parity: extend the mock to stream
delta.audiochunks and assert the gateway forwards them. - Real-chain: a streaming variant of the AISIX-Cloud PR-K (#677) gpt-4o-audio spec.
Refs
- non-stream fix: #517
- product issue: api7/AISIX-Cloud#684
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with OpenAiStreamDelta in crates/aisix-provider-openai/src/wire.rs, then trace ChatDelta and RenderedDelta in crates/aisix-proxy/src/render.rs alongside the non-stream fix in #517. Extend the mock-parity coverage and the streaming AISIX-Cloud spec so incremental delta.audio chunks are forwarded to the client.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100