openai / openai/codex

Raw chain-of-thought leaked into visible output as derailment gibberish, then turn aborted with invalid_prompt usage-policy error (gpt-5.6-sol high)

Open
#43,165 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug CLI model-behavior
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

What version of Codex CLI is running?

codex-cli 0.147.0 (TUI) at the time of the session; the machine has since been upgraded to 0.153.4.

What subscription do you have?

Pro (20x)

What platform is your computer?

Linux (Ubuntu, kernel 5.15.0-160-generic x86_64), terminal: bash

Which model were you using?

gpt-5.6-sol, effort: high (the session started on gpt-6-astra and was on gpt-5.6-sol for the failing turn; last token_count shows model_context_window: 258400)

What issue are you seeing?

At the end of a very long autonomous session (~16 h wall time, 1035 sampling commits, ~80M cumulative input tokens), the model began streaming its final summary message and the visible output derailed into a mix of valid Chinese answer text, raw English chain-of-thought meta-commentary, and off-distribution garbage tokens. Sanitized excerpt of what was rendered in the TUI:

  • b43c30cb (nested OpenSSD): FTLлуг? Wait mismatch memory says bnant. Need correct b multiline.
  • 822ac5 Sandboxelassen: persistent decode, Wbots. Need exactly blæ.
  • 472fc21: current evidence.

Tests etc.

Need ensure no typo. Let's formulate Chinese. Mention actual boundary not solved:
[...]
"结束" could mean all follow-on boundary tasks? We say

The stream was cut off mid-sentence ("We say") and the turn terminated with:

Invalid prompt: your prompt was flagged as potentially violating our usage policy. Please try again with a different prompt: https://platform.openai.com/docs/guides/reasoning#advice-on-prompting

The user's prompts contained nothing policy-relevant; this was a hardware/RTL verification task (OpenSSD FTL work). The quoted fragments ("Need correct b multiline", "Let's formulate Chinese", "'X' could mean ...? We say") are plainly the model's raw hidden reasoning, not user-authored content.

Rollout-file forensics

I inspected the session rollout JSONL afterwards:

  • The garbled message was never persisted. The rollout only commits completed response_items (each followed by a spine_transition/sampling_commit record); the aborted stream left no message item. The tail of the file is: reasoning item with encrypted_content only (empty summary) at 04:09:13Z → task_complete with the invalid_prompt error at 04:09:17Z.
  • The last persisted assistant message (04:08:47Z) is a normal in-progress update.
  • Nothing in the preceding turns contains policy-flaggable content.

So the failure chain appears to be: sampling derailment while the model was trying to reproduce commit hashes verbatim (it visibly struggled: "memory says bnant. Need correct b") → raw analysis-channel text spilled into the visible message channel → the output-side detector flagged the leaked reasoning and aborted the stream mid-message.

Relation to #34245

This looks like the same failure chain as #34245 (same model family, gpt-5.6 Sol High; reasoning text visible in TUI; identical terminal invalid_prompt error), but with a different trigger: in #34245 the derailment followed broken tool-call state after the cwd was moved; here there was no tool-call failure — the derailment happened spontaneously mid-message during hash reproduction in a long session. Filing separately in case the trigger distinction matters; feel free to dedupe.

What steps can reproduce the bug?

No reliable reproducer; observed once. Hypothesized contributing factors: (1) very long session with ~1000+ sampling commits, (2) the model attempting to emit exact commit hashes (low-probability token sequences), (3) final-answer composition in a non-English language while reasoning in English.

What is the expected behavior?
  • Raw reasoning should never enter the visible message channel, regardless of sampling state.
  • If the output derails, the response should be regenerated or aborted with an honest internal-error message — not a usage-policy error that attributes the leak to the user's prompt.
  • Ideally the aborted partial message should not be rendered at all.
Additional information
  • Session ID: 01a0716f-3580-7130-9335-2b0c82f6e638
  • Error timestamp: 2026-09-06T04:09:17Z (turn started ~03:32Z)
  • The full rollout file contains ~1000 encrypted_content reasoning blocks; given the findings of arXiv:2608.09867 (encrypted reasoning blocks were replay-decodable before the August 2026 fixes), I am not attaching it publicly. Happy to share it privately with maintainers on request.
  • This happened in an unattended/bypass-approvals configuration, so the leaked-then-flagged turn ended the session without user awareness until later inspection.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Codex CLI TUI streaming path and the rollout JSONL records named in the report: response_item, spine_transition/sampling_commit, and task_complete. Compare the persisted tail with what was rendered, focusing on the boundary between reasoning and visible-message channels and handling of invalid_prompt after a partial stream. Done means raw reasoning is never rendered and derailments produce an honest internal error without misleading policy attribution.

Written by the indexing model from the issue text.

Assessment

Tech stack
bash, linux, rust
Domain
ai, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.