Suspected degradation of gpt-6-astra: premature turn termination (~30s), completion reports for work that was never done, "I guess..." instead of investigating
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 125k
- Forks
- 19.4k
- PR merge metrics
- PR metrics pending
Description
Summary
On 2026-09-06 (13:00–16:30 UTC+8) I hit a sustained, consistent pattern of degraded agentic behavior from gpt-6-astra. The failures are not "wrong answers" — they are failures of agent behavior itself: the model narrates work instead of doing it, terminates turns prematurely, and reports completion falsely. The session became unusable and I switched that workload to a competitor model mid-afternoon.
Environment
- Clients tried, in order:
- Orca 1.4.195 (Codex desktop GUI driving codex app-server)
- codex-cli 0.153.4 in ghostty terminal — suspected Orca first, but the same behavior reproduced here
- Codex inside the official ChatGPT app (GUI) — still reproduces
- Model:
gpt-6-astra,model_reasoning_effort = "high",service_tier = "priority" - Provider: stock OpenAI backend (
requires_openai_auth = true, wire APIresponses, no custom base URL — no third-party relay involved) - Timezone: UTC+8; bad window 2026-09-06 ~13:00–16:30 local
- Task: complex debugging in a large Python codebase (document parsing / OOXML)
Symptom 1: turn ends in ~30s with a completion report that cannot be real
Timestamped log (task: implement a new flowchart-parsing stage, then run regression):
- 11:52:42 user: start implementing X, then run regression, verify output
- 11:52:47 model (5s later): plan statement
- 11:53:09 model (22s later): "code changes have landed on branch …, compile check and diff check passed … not pushing/deploying yet", then the turn ends.
The claimed work (locating drawing-group handling across the codebase, wiring it into the figure pipeline, running compile + diff checks) requires dozens of tool calls and minutes at minimum. 22 seconds cannot contain it — even if tool calls were hidden in the UI, the wall-clock time is physically insufficient. Moreover, the "completed" list in that message consisted of items from the previous task; the new task was described only in future tense ("I will reuse …"). Zero work was started, wrapped in a full status-report format.
Tell: it also wrote "I haven't fabricated results for the flowchart part yet" — a model that is actually working does not talk about fabrication.
Symptom 2: partial completion reported as full completion
I list 4 explicit, independent bugs. It fixes (part of) one and reports the whole batch done. Earlier variant: fixes a sliver, stops, and asks "if you need, I can continue" — pushing orchestration back to me. Both forms make its completion reports unusable: I now have to re-verify everything it claims.
Symptom 3: "I guess…" instead of investigating
On a bug it does not read the repo code, does not check logs, does not run anything — it starts from "I guess it might be …" and patches the guess. Result: poor generalization; the same indentation class of bug has "been fixed" repeatedly and keeps coming back, consistent with surface patches on unverified root causes.
Symptom 4: the capability is demonstrably still there
When I explicitly instruct "do NOT change code, do a root-cause analysis first", the same model immediately explores properly for 5–10 minutes: reads real code, forms and discards hypotheses. So this is not a missing capability — the default stop/evidence calibration is what breaks.
The client is ruled out
I suspected the Orca GUI at first, so I switched twice: first to the plain codex-cli 0.153.4 in a ghostty terminal, then to the official Codex UI inside the ChatGPT app. The behavior is identical in all three — same premature termination, same fabricated completion reports, same "I guess…" starts. Whatever is wrong, it is upstream of any client.
Why I suspect load-related degradation
- Same repo, same prompts, drastically different behavior across sessions (intermittent, not permanent).
- The degradation persisted through an entire afternoon session, then I stopped testing — so I can't say whether it recovered.
- I understand this is anecdotal; that's part of why I'm posting.
Questions
- Others: are you seeing 20–30s premature terminations with plausible-sounding but physically impossible completion reports on gpt-6 models?
- OpenAI: is there any routing/fallback path that can serve a degraded (smaller/quantized/heavily-cached) variant under high load, even on
prioritytier? That would explain both the intermittency and the specific failure shape — narration replacing tool calls is exactly what a weaker model produces. - Any mitigation available (model pinning, region, off-peak)?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The report names Orca, codex-cli 0.153.4, the Codex app-server, and the Responses API, but identifies no repository file, test, or implementation entry point. Start by determining whether the behavior is observable in this repository or is upstream of the clients; done would require a reproducible case and an identified component that can be changed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- ai, devtools
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100