openai / openai/openai-agents-python

Realtime: close() keeps the previous connection's item and audio state, so a reconnect on the same model truncates and retrieves stale item ids

Open
#5,068 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
29.6k
Forks
4.8k
Avg merge
1d 20h
Merged PRs (30d)
123

Description

Please read this first
  • Have you read the docs? Yes.
  • Have you searched for related issues? Yes. #4189 covers cleanup after a failed connection attempt and #4461 covers ending iteration after a clean server close; neither covers state that survives a successful close into the next connection.
Describe the bug

OpenAIRealtimeWebSocketModel.close() clears the response-scoped audio indexes but leaves three per-connection values in place: _current_item_id, the ModelAudioTracker (its last audio item and per-item states), and _created_session. connect() accepts a new connection on the same instance after close(), and RealtimeRunner reuses one model instance across runner.run() calls, so the second session starts with the first session's item ids and turn-detection settings.

On that second connection, before any assistant audio has been produced:

  • the first input_audio_buffer.speech_started emits RealtimeModelAudioInterruptedEvent(item_id="<old item>") and sends conversation.item.truncate for an item id the new server session has never seen;
  • the first conversation.item.input_audio_transcription.completed sends conversation.item.retrieve for that same old item id;
  • RealtimeModelSendInterrupt does the same as the first bullet;
  • until the new session.created arrives, _send_interrupt decides whether to cancel the response from the previous session's turn_detection.interrupt_response.

The server answers the foreign-id messages with error events, which reach the application as RealtimeError, and the application also receives an audio_interrupted for an item it cannot map to anything in the current session.

Debug information
  • Agents SDK version: main at 9f6f6b10 (also present in v0.22.2)
  • Python version: 3.12.13
  • Operating system: macOS 15
  • Model and model provider: OpenAIRealtimeWebSocketModel (OpenAI Realtime API over WebSocket)
  • Does the issue reproduce with the latest Agents SDK release? Yes
  • Does the issue occur consistently or intermittently? Consistently
Repro steps

Self-contained, no network: the WebSocket factory is replaced with a recorder.

import asyncio, base64, json
from agents.realtime.model_events import RealtimeModelAudioInterruptedEvent
from agents.realtime.openai_realtime import OpenAIRealtimeWebSocketModel


class RecordingWebSocket:
    def __init__(self):
        self._closed = asyncio.Event()
        self.sent = []

    def __aiter__(self):
        return self

    async def __anext__(self):
        await self._closed.wait()
        raise StopAsyncIteration

    async def send(self, payload):
        self.sent.append(json.loads(payload))

    async def close(self):
        self._closed.set()


SESSION_CREATED = {
    "type": "session.created", "event_id": "ev", "session": {
        "type": "realtime", "model": "gpt-realtime",
        "audio": {"input": {"turn_detection": {"type": "semantic_vad", "interrupt_response": True}},
                  "output": {"format": {"type": "audio/pcm", "rate": 24000}}},
    },
}


async def main():
    model = OpenAIRealtimeWebSocketModel()
    events = []

    class Listener:
        async def on_event(self, event):
            events.append(event)

    model.add_listener(Listener())
    sockets = []

    async def fake_connect(*args, **kwargs):
        sockets.append(RecordingWebSocket())
        return sockets[-1]

    model._create_websocket_connection = fake_connect

    # First session: one assistant audio item, then close.
    await model.connect({"api_key": "test", "initial_model_settings": {}})
    await model._handle_ws_event(SESSION_CREATED)
    await model._handle_ws_event({
        "type": "response.output_audio.delta", "event_id": "e1", "response_id": "resp_old",
        "item_id": "item_old", "output_index": 0, "content_index": 0,
        "delta": base64.b64encode(b"\x00\x01" * 2400).decode(),
    })
    await model._handle_ws_event({"type": "response.done", "event_id": "e2", "response": {"id": "resp_old"}})
    await model.close()
    events.clear()

    # Second session on the same instance: user starts speaking before any assistant audio.
    await model.connect({"api_key": "test", "initial_model_settings": {}})
    await model._handle_ws_event(SESSION_CREATED)
    await model._handle_ws_event({"type": "input_audio_buffer.speech_started", "event_id": "e3",
                                  "audio_start_ms": 0, "item_id": "item_user_new"})
    await model.close()

    print([e for e in events if isinstance(e, RealtimeModelAudioInterruptedEvent)])
    print([m for m in sockets[1].sent if m["type"] == "conversation.item.truncate"])


asyncio.run(main())

Output on main:

[RealtimeModelAudioInterruptedEvent(item_id='item_old', content_index=0, type='audio_interrupted')]
[{'audio_end_ms': 0, 'content_index': 0, 'item_id': 'item_old', 'type': 'conversation.item.truncate'}]
Expected behavior

Both lists are empty. A connection opened after close() should start with no item, audio, or session state from the previous connection, the same way close() already clears the response audio indexes. I have a fix with regression tests ready and will open a PR referencing this issue.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with OpenAIRealtimeWebSocketModel.close() and connect(), then run the self-contained recorder reproduction described in the issue. Verify that a second connection emits no stale audio_interrupted event or conversation.item.truncate message, and add or run regression tests covering reuse after close.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
audio-video-rtc
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.