openai / openai/openai-python

Blocking the event loop since 3.x ?

Open
#3,777 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
31.6k
Forks
5.7k
Avg merge
1d 6h
Merged PRs (30d)
96

Description

Confirm this is an issue with the Python library and not an underlying OpenAI API
  • This is an issue with the Python library
Describe the bug

We run a service using a standard asyncio event loop, running various async code like aiokafka, redis (async) or aiohttp. After upgrading to 3.x version, we notice after a while symptoms that the event loop is blocked:

  • timeout accessing redis
  • timeout with the aiokafka coordinator/heartbeat co-routine

We tried using the default httpx2 client or httpx2 aiohttp client, but it doesn't solve the issue. Reverting to 2.54 solves the issue

Sorry to be so vague, we have nothing in our logs that could give a pointer to a more precise section of the code. We do use chat completion APIs, and structured output with Pydantic

To Reproduce
  1. Update to 3.x
  2. Wait few hours, the CPU usage is raising, symptom of blocked event loop appears
Code snippets

OS

Linux x86

Python version

Python v3.12.3

Library version

OpenAI v3.6.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the upgrade from version 2.54 to 3.6.0 in a Python asyncio service using chat completions or structured outputs, while monitoring CPU and event-loop responsiveness. Compare the default httpx client with the httpx aiohttp client and verify whether Redis, aiokafka, or other async operations still time out after several hours.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, redis
Domain
backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.