openai / openai/codex

GPT-5.6 Luna returns silent empty successful stops on large contexts with no error signal (quota still consumed)

Open
#37,879 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug context model-behavior rate-limits
Dominant language
Rust
Stars
125k
Forks
19.5k
PR merge metrics
PR metrics pending

Description

Summary

On large agent conversations (~360K–370K input tokens, ~570+ messages), the ChatGPT subscription / Codex backend for GPT-5.6 Luna (reasoning effort max) returns HTTP 200 completions with:

  • finish_reason / stop reason: stop
  • assistant text: none / null
  • tool calls: none
  • top-level usage: missing / empty
  • no error object
  • no incomplete_details
  • usage/quota endpoint still reports not limit-reached during the empty streak

Clients therefore treat the turn as a normal successful stop. The UI appears stalled, while preceding large turns (and some empty turns that still emit encrypted reasoning) continue to consume subscription quota.

This looks related to #32389 (Terra empty successful final response after tool use), but here the failure becomes a hard cliff on Luna at very large context: multiple consecutive empty stops, not just an intermittent mid-task empty final.

Observed response shape

After a streamed Responses turn, the effective Chat Completions-shaped result was:

{
  "object": "chat.completion",
  "model": "gpt-5.6-luna",
  "choices": [
    {
      "index": 0,
      "finish_reason": "stop",
      "message": {
        "role": "assistant",
        "content": null
      }
    }
  ]
}

Two variants were captured in the same session:

  1. Empty stop with encrypted reasoning item(s) whose summary and content arrays were empty (private reasoning present, no user-visible output, no tools).
  2. Empty stop without reasoning, text, or tools (completely barren).

In both cases there was still no machine-readable failure signal.

Reproduction context

  • Auth path: ChatGPT subscription / Codex backend
  • Model: gpt-5.6-luna
  • Reasoning effort: max
  • Conversation size at failure: about 572–578 messages, about 19 tools present in the request
  • Input size near failure: about 371K tokens on the last useful turn
  • Immediately before the cliff, turns were still returning tool calls as context climbed through roughly 300K–370K tokens
  • After the cliff, several consecutive turns returned the empty-stop shape (latency about 5–22s each)
  • Retrying the same oversized conversation reproduced the same empty stop
  • Quota meter did not report limit-reached during the empty streak

Detection signature:

  • transport success (200)
  • stop/finish reason stop
  • no assistant text
  • no tool calls
  • empty/missing usage
  • no error / incomplete payload

Why this is harmful

  1. No failure signal – clients cannot distinguish “model finished” from “model produced nothing”, so agent loops stall instead of erroring.
  2. Quota burn – large preceding tool-loop turns at 300K+ input consume subscription allowance; empty successful stops still look billable/completed from the client’s point of view.
  3. Retry amplification – because the response looks like a successful empty completion rather than an error, clients may keep retrying the same oversized chat and waste more quota.

Expected behaviour

At least one of:

  1. Return a non-200 or an explicit error / incomplete status when a turn yields no text and no tool calls after the request was accepted, or
  2. Include a machine-readable reason (for example context-too-large / generation-aborted), or
  3. Do not charge / do not advance quota for turns that yield no user-visible output and no tool calls.

Prefer failing loudly over returning a successful stop with null/empty content.

Related

  • #32389 — GPT-5.6 Terra intermittently returns an empty successful final response after tool use
  • Same empty-stop class previously seen on GPT-5.6 Sol around ~360K-token contexts (HTTP 200, stop, null content, empty usage, quota not exhausted)

Ask

Please confirm whether silent empty successful stops on oversized Codex/ChatGPT subscription contexts are intended. If not, please treat missing failure signalling + quota consumption on empty generations as a backend bug.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No repository file, test, or entry point is identified. Start by reproducing the empty-stop response using the stated model, reasoning effort, conversation size, and detection signature; done would mean confirming the behavior and defining an explicit error, incomplete status, or non-charging outcome.

Written by the indexing model from the issue text.

Assessment

Domain
ai-infra-agents, backend
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.