simonw / simonw/llm

Allow longer OpenAI Responses requests before timing out

Open
#1,504 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
12.5k
Forks
998
Avg merge
6d 12h
Merged PRs (30d)
15

Description

In 0.32a3, OpenAI Responses SDK times out by default after 10 minutes and retries twice, leading to 30 minutes wait before time out error for long API calls (e.g. gpt-5.5-pro xhigh with max_tokens > 25,000).

Suggested fix is introducing --timeout and --max-retries model options for Response models. Example agent implementation that works for me is at 6519b1a06a5c957c9d41de44db0e8e2d240eba1e

Details

I'm using llm 0.32a3 with an extra OpenAI model configured for gpt-5.5-pro:

- model_id: gpt-5.5-pro
  model_name: gpt-5.5-pro
  responses: true
  reasoning: true
  vision: true
  supports_tools: true
  supports_schema: true
  can_stream: false

I'm trying to run it with a template that gives it a large output budget:

model: gpt-5.5-pro
options:
  reasoning_effort: xhigh
  max_tokens: 25000

This currently works for me with max_tokens: 12500, but 25000 and above consistently time out after what seems to be exactly 30 minutes:

cat prompt | llm -t gpt-pro-max > out.pro
Error: Request timed out.

After adding model --timeout and --max-retries options (6519b1a06a5c957c9d41de44db0e8e2d240eba1e) and changing the template to the following, I have had a successful 17 minutes command run.

model: gpt-5.5-pro
options:
  reasoning_effort: xhigh
  max_tokens: 50000
  max_retries: 0
  timeout: 7200 # two hours

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the Response model implementation and the referenced example commit 6519b1a06a5c957c9d41de44db0e8e2d240eba1e. Trace how model options are exposed to the CLI, then verify that Response models accept configurable timeout and max-retries values and that long requests can use them without the current default limits.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, cli
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.