openai / openai/openai-agents-python

Voice: on Python 3.10 a streamed STT session-setup timeout escapes as asyncio.TimeoutError instead of STTWebsocketConnectionError

Open
#5,074 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
29.6k
Forks
4.8k
Avg merge
1d 20h
Merged PRs (30d)
123

Description

Please read this first
  • Have you read the docs? Yes.
  • Have you searched for related issues? Yes. #4593 moved the STT deadline to a monotonic clock and #4170 covers listener errors; neither covers the exception class of the asyncio.wait_for timeout.
Describe the bug

_wait_for_event in agents/voice/models/openai_stt.py raises the builtin TimeoutError when its deadline has already elapsed, but the ordinary timeout comes from asyncio.wait_for. On Python 3.10 that raises asyncio.TimeoutError, which is a different class from the builtin (they became the same object in 3.11). The two except TimeoutError clauses in _setup_connection only catch the builtin, so on 3.10 a session.created or session.updated event that never arrives falls through to the generic handler and reaches the caller as a bare asyncio.TimeoutError with an empty message, instead of the documented STTWebsocketConnectionError("Timeout waiting for transcription_session.created event").

Code that handles STTWebsocketConnectionError (the type agents.voice.exceptions documents for connection failures) misses the failure on 3.10. The existing test_timeout_waiting_for_created_event does not catch this because it patches monotonic, which exercises only the deadline branch that raises the builtin.

Debug information
  • Agents SDK version: main at 58a6d2c9 (also present in v0.22.2)
  • Python version: 3.10.20 (not reproducible on 3.11+, where asyncio.TimeoutError is TimeoutError)
  • Operating system: macOS 15
  • Model and model provider: OpenAISTTTranscriptionSession (streaming transcription over WebSocket); reproduced with a mocked WebSocket that never sends session.created
  • Does the issue reproduce with the latest Agents SDK release? Yes
  • Does the issue occur consistently or intermittently? Consistently on 3.10
Repro steps

Self-contained, no network. Run with Python 3.10.

import asyncio
from unittest.mock import AsyncMock, patch

import numpy as np
from openai import AsyncOpenAI

from agents.voice import OpenAISTTTranscriptionSession, StreamedAudioInput, STTModelSettings
from agents.voice.exceptions import STTWebsocketConnectionError


async def main():
    with patch("agents.voice.models.openai_stt.SESSION_CREATION_TIMEOUT", 0.01):
        mock_ws = AsyncMock()
        mock_ws.__aenter__.return_value = mock_ws
        mock_ws.__aiter__.return_value = iter([])  # The server never sends session.created.
        with patch("websockets.connect", return_value=mock_ws):
            audio_input = StreamedAudioInput()
            await audio_input.add_audio(np.zeros(2400, dtype=np.int16))
            session = OpenAISTTTranscriptionSession(
                input=audio_input,
                client=AsyncOpenAI(api_key="FAKE_KEY"),
                model="whisper-1",
                settings=STTModelSettings(),
                trace_include_sensitive_data=False,
                trace_include_sensitive_audio_data=False,
            )
            try:
                async for _ in session.transcribe_turns():
                    pass
            except STTWebsocketConnectionError as error:
                print("STTWebsocketConnectionError:", error)
            except Exception as error:
                print("escaped as", type(error).__module__ + "." + type(error).__qualname__, repr(str(error)))
            finally:
                await session.close()


asyncio.run(main())

Output on main with Python 3.10.20:

escaped as asyncio.exceptions.TimeoutError ''
Expected behavior

The same as on Python 3.11 and later:

STTWebsocketConnectionError: Timeout waiting for transcription_session.created event

I have a fix with regression tests ready and will open a PR referencing this issue.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read agents/voice/models/openai_stt.py, focusing on _wait_for_event and _setup_connection, then inspect test_timeout_waiting_for_created_event. Run the existing test and add regression coverage for the asyncio.wait_for path on Python 3.10; done means missing session.created or session.updated events produce STTWebsocketConnectionError with the documented message.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
audio-video-rtc
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.