[Regression] Codex Desktop: Astra/Sol quality regressions, excessive context replay, and long-thread capacity failures

Summary

I am seeing a severe output-quality regression in Codex Desktop when using gpt-5.6-sol with High reasoning on a very small, clean visual task.

The same simple task produces a much more detailed and coherent result in ChatGPT web, while Codex spends much longer, performs unnecessary agent/tool work, replays a very large amount of context, and still produces a substantially simpler artifact.

This report is not based only on subjective output quality. I inspected the Codex session JSONL and confirmed that the Codex session was actually configured as gpt-5.6-sol + High, with no explicit server_overloaded error in this Sol run.

Related: #41222, #41450.

Codex version

• Codex Desktop
• CLI version from session log: 0.155.0-alpha.9.2
• Platform: macOS

Reproduction

Start a fresh/empty workspace and select:

• Model: gpt-5.6-sol
• Reasoning effort: high

Use this exact prompt:

创建一个HTML，内容是SVG绘制一个鹈鹕骑摩托车的2D动画，你不需要任何测试创建一个HTML

English meaning:

Create one HTML file containing a 2D animation of a pelican riding a motorcycle, drawn with SVG. You do not need to do any testing; just create the HTML.

Expected behavior

For such a small task, I would expect Codex to:

1. create the requested single HTML/SVG artifact directly;
2. spend most of the model budget on the actual SVG composition and animation;
3. avoid browser/environment validation because the prompt explicitly says testing is unnecessary;
4. produce output quality reasonably consistent with the selected high-capability model and High reasoning mode.

Actual behavior

In Codex Desktop, the run:

• spent several minutes on agent/tool activity;
• loaded frontend skill/runtime context;
• performed browser/environment preview/validation despite the explicit “no testing” instruction;
• produced a relatively simple SVG page (about 323 lines in this run);
• was visually much less detailed than the result produced by ChatGPT web for the same task.

The quality gap was especially obvious in the subject itself: the Codex version used a simplified symbolic motorcycle/pelican composition, while the web result contained substantially more detailed motorcycle geometry, mechanical structure, rider pose, background depth, and interaction controls.

GPT-6 Astra shows the same regression pattern

I also reproduced the same exact prompt in Codex Desktop using GPT-6 Astra, at both Light and High reasoning settings.

Observed Astra behavior:

• Astra Light: the run took about 3m05s and produced a very simple SVG scene with low visual/detail complexity.
• Astra High: the run took about 5m11s. Despite the prompt explicitly saying “you do not need any testing”, Codex still spent time checking browser availability, opening/previewing the animation, and inspecting the rendered page.
• The Astra High result was somewhat more elaborate than Light (city silhouette, road/parallax elements, wheel rotation, simple pelican motion, pause/resume behavior), but it was still far below the level of visual detail expected from a multi-minute High-reasoning run.
• The same task in ChatGPT web produced a dramatically richer composition: much more detailed motorcycle geometry/mechanical structure, better rider pose, denser scene construction, and more polished controls.

The Astra reproduction therefore suggests this is not limited to GPT-5.6 Sol. The common factor is Codex Desktop / the Codex agent runtime.

For precision: the Sol High reproduction below is the one I matched directly to a JSONL session and token accounting. The Astra Light/High visual reproductions are documented from the Codex UI runs/screenshots; I have not yet matched the exact Astra High visual run to a separate JSONL session in this report.

There is also an independent Astra session in my logs that failed with:

Selected model is at capacity. Please try a different model.
codex_error_info: server_overloaded

That capacity event is not the same as the Sol High SVG run and should not be treated as proof of fallback for the affected Sol request. It does, however, show that Astra capacity pressure was present during the same period.

Related Astra report: #42937.

Session evidence for GPT-5.6 Sol High

The relevant Codex session log confirms that this was not simply a UI label mismatch:

"model": "gpt-5.6-sol"
"reasoning_effort": "high"
"effort": "high"
"model_provider": "openai"

The cumulative token accounting near the end of this small task was:

input_tokens:              887,076
cached_input_tokens:       827,264
output_tokens:               9,230
reasoning_output_tokens:     1,468
model_context_window:       258,400

A later model call in the same session carried approximately:

input_tokens:   65,589
cached_tokens:  64,640

for a task whose user prompt is only a short sentence.

This suggests that a very large Codex agent/harness/tool context is being replayed across the loop even for a trivial task.

The session initialization also included a very large skills/tool/runtime catalog unrelated to this task (iOS, macOS, Cloudflare, Hugging Face, patent tooling, data analytics, Unity, Canva, Notion, PDF, spreadsheets, etc.).

Important control: no explicit overload in the affected Sol run

I searched the session logs for capacity errors.

There was the separate GPT-6 Astra session described above that failed with server_overloaded.

However, I did not find a corresponding server_overloaded event for this gpt-5.6-sol High SVG run.

So the simplest explanation “High silently became Low because this request hit an explicit capacity error” does not fit the observed Sol session.

Why this looks like a Codex runtime / harness regression

The evidence points to at least one of these possibilities:

1. excessive context replay / agent harness bloat;
2. degraded orchestration or premature satisficing;
3. model budget being spent disproportionately on tool-loop overhead rather than task execution;
4. a server-side serving/routing change that is not visible in the client logs.

The fact that both GPT-6 Astra and GPT-5.6 Sol show the same qualitative failure pattern makes a Codex-runtime/orchestration issue particularly plausible, although a shared serving/routing issue cannot be ruled out.

The current local logs show the configured/requested model, but do not expose enough server-side information to rule out a silent serving change.

It would be very useful if Codex diagnostics exposed fields such as:

requested_model
resolved_model
model_snapshot
deployment_id / serving_pool
backend_revision

That would make it possible to distinguish a harness regression from a server-side routing/serving regression.

Why I am filing separately from related reports

• #41222 reports a broader GPT-5.6 Sol behavioral regression.
• #41450 reports reproducible token amplification / scheduling inefficiency.
• #42937 reports reduced autonomous completion / reliability across GPT-5.6 Sol and GPT-6 Astra in Codex.

This report adds a small, clean, reproducible visual artifact benchmark where:

• the same exact prompt reproduces poor behavior in both GPT-6 Astra and GPT-5.6 Sol inside Codex;
• GPT-5.6 Sol is confirmed as gpt-5.6-sol in the JSONL;
• High reasoning is confirmed in the Sol session log;
• there is no explicit overload in the affected Sol run;
• cumulative context replay is measurable;
• the final Codex artifacts are visibly much worse than the same task outside Codex.

I can provide the full JSONL session and screenshots of the Astra Light, Astra High, Sol High, and ChatGPT web outputs if maintainers want them.

---

Update: real engineering task shows instruction-following regression and long-thread capacity failure

I have now reproduced a second failure mode on a real repository task, beyond the small SVG benchmark.

Environment for this reproduction:

• Codex Desktop app version: 26.915.31945
• Platform: macOS
• ChatGPT plan: 5x
• Account region: US
• Model: GPT-6 Astra (Medium)

1. Initial behavior: shallow task modeling / premature implementation

The task was an architecture-level refactor of a model/provider/skill configuration system.

The user explicitly asked Codex to:

• review the previous Codex thread;
• inspect the whole runtime architecture, not only the admin UI;
• design extensible Provider / Model / Skill contracts;
• support provider discovery, model parameters, capabilities, defaults/fallbacks, migration, credential handling, and future extensibility;
• avoid treating the task as a few frontend fields.

Despite this, Codex initially converged too early on a narrow implementation path and began modifying the database schema/UI before it had completed the requested architecture review.

A representative early change was effectively just adding fields such as display_name to the existing model table.

The behavior changed only after the user explicitly forced Codex to stop all writes and required a full architecture audit before implementation.

After that stronger instruction, Codex successfully:

• read the previous thread;
• inspected Provider, ModelManager, capability registry, skill runtime sources, configuration files and planning documents;
• identified four conflicting configuration sources;
• identified that resolve_route() was not the single runtime entry point;
• identified that skill revisions existed but were not actually consumed by runtime;
• produced a coherent target architecture around Provider Registry, Model Registry, parameter schemas, route policies, Runtime Resolver, Skill Registry and Skill Runtime.

This suggests that the model still has the underlying capability, but the default Codex behavior is currently much more prone to:

• premature convergence;
• under-scoping;
• satisfying the minimum visible implementation;
• beginning edits before completing the requested system-level investigation.

The task quality improved substantially only when the prompt imposed explicit gates such as "do not edit until the architecture audit is complete" and enumerated the full dependency surface.

2. With explicit execution constraints, Astra began performing normally

After the architecture plan was converted into a strict execution specification, Astra Medium began a much more appropriate staged implementation.

It:

• reverted the earlier temporary local patches;
• created a versioned migration layer;
• created credential-store abstraction;
• created provider adapter and model resolver layers;
• modified ModelManager to reduce old hard-coded logic;
• created Skill Runtime;
• started wiring Admin API and runtime consumers;
• investigated duplicate resolver/adapter implementations and shared module placement.

At the point of interruption, the working tree contained approximately:

10 files changed
+1,020
-82

This is useful evidence that the model was capable of sustained, coherent implementation once the task scope and stopping conditions were made explicit.

3. The long-running thread was then repeatedly killed by capacity errors

After approximately 15m30s of active work, the same GPT-6 Astra thread began returning:

Selected model is at capacity. Please try a different model.

This happened repeatedly. Automatic retries also failed, and the task became stalled.

The important detail is that the failure happened after a long, successful multi-tool coding run, not at session start.

The working tree remained intact; only further model inference stopped.

4. Same account and same model could still answer a trivial request immediately

While the long engineering thread was repeatedly failing with Selected model is at capacity, I opened another Codex conversation on the same account and selected the same GPT-6 Astra model.

A trivial message:

你好

received an immediate normal response.

This makes an account-wide entitlement failure unlikely.

5. A new thread on the same workspace/model could resume the complex task

I then opened a new Codex thread, kept the same account, same GPT-6 Astra model, and same working tree, and gave it a short handoff telling it to:

• read the previous thread;
• inspect git status and the uncommitted diff;
• recover the implementation state;
• continue from the interruption point instead of restarting.

The new thread worked normally.

Within about 1–2 minutes it:

• read the previous conversation;
• inspected the complete uncommitted diff;
• correctly recovered that migration, registry, credential store, provider adapters, resolver, skill runtime and API work were already partially implemented;
• resumed investigating canonical resolver/adapter placement and import-path cleanup.

So the observable sequence was:

Same account + Astra + long thread
    -> repeated "Selected model is at capacity"
    -> automatic retries fail

Same account + Astra + fresh trivial thread
    -> works immediately

Same account + Astra + fresh thread + same repository/working tree
    -> complex engineering task works again

6. This suggests a thread/context-sensitive capacity or scheduling problem

I cannot see the server-side scheduler, so this is a hypothesis rather than a confirmed root cause.

However, this pattern is consistent with a problem where a long Codex thread becomes substantially more expensive to admit/serve because it carries a large amount of conversation/tool context, cached context, diffs and agent state.

A fresh thread with a compact handoff may require a much smaller active context and can therefore be scheduled successfully even while the older thread is repeatedly rejected.

Possible areas to investigate:

• request admission behavior as active thread context grows;
• KV-cache / long-context serving pressure;
• whether a thread becomes pinned to an overloaded serving pool;
• whether Codex Desktop retries the same overloaded backend/session route;
• whether long-running tasks should be transparently rescheduled;
• whether a compacted/handoff continuation should be offered automatically after repeated capacity failures;
• whether Desktop should distinguish "model globally unavailable" from "this request/thread cannot currently be admitted".

7. The combined symptoms now look broader than a single model-quality issue

The evidence observed so far is now:

1. GPT-6 Astra and GPT-5.6 Sol output-quality regression on the same small visual benchmark.
2. Confirmed Sol High configuration in local JSONL, but unexpectedly large cumulative context replay and relatively little reasoning output.
3. Instruction-following / task-scope regression on a real architecture task: Codex began local implementation before completing the explicitly requested system-level audit.
4. Behavior improves when the user adds unusually explicit scope, persistence, and "do not implement yet" gates.
5. Astra capacity failures can interrupt a long-running task after 15+ minutes of successful work.
6. The same account/model still handles trivial requests immediately.
7. A fresh thread on the same working tree can resume the same complex task successfully.

Taken together, this looks more like a combination of Codex orchestration/context-management/serving behavior than a simple "the selected model is unavailable" event.

Requested diagnostics / product improvements

It would help enormously if diagnostics exposed:

requested_model
resolved_model
reasoning_effort
model_snapshot
serving_pool / deployment_id
backend_revision
active_context_tokens
cached_context_tokens
compaction state
capacity rejection reason
retry target / serving pool

It would also be valuable if Codex Desktop could recover long-running work by automatically creating a compact continuation/handoff when repeated server_overloaded / capacity errors make the current thread unschedulable.

I can provide screenshots showing the repeated capacity failures, the preserved 10-file working-tree diff, the successful trivial request, and the successful fresh-thread continuation if useful.

Start by reproducing the exact prompt in Codex Desktop/CLI 0.155.0-alpha.9.2 and inspecting the GPT-5.6 Sol session JSONL, including model, reasoning, token, and overload fields. Compare the Sol and Astra runs to determine whether context replay, tool-loop overhead, or serving behavior explains the regression; done means the responsible layer is identified and the observed behavior is consistently characterized.