GPT-6 Astra regresses exploratory problem formation vs GPT-5.2/5.5/5.6 Sol in complex engineering work

Summary

I am a very heavy Codex user working on a large custom Unity RPG system. I have used multiple generations of GPT/Codex extensively on the same long-running project, so I have a fairly stable longitudinal baseline for model behavior.

I am reporting what I believe is a regression in a capability that is difficult to capture with normal coding benchmarks:

problem formation, exploratory search, framing resistance, and causal hypothesis revision.

This is not primarily a complaint about coding quality, speed, instruction following, or benchmark intelligence. GPT-6 Astra is an extremely capable executor. It is fast, precise, disciplined, and often excellent once the problem has already been correctly defined.

My concern is that it appears substantially worse at helping discover what the real problem is when the initial user framing is incomplete or wrong.

For my workflow, this is a critical capability rather than a stylistic preference.

My workflow: the prompt is often a probe, not a specification

A large part of my work with Codex is closer to a research lab than traditional ticket-based software development.

I frequently do not know the correct solution when I write the first prompt.

The prompt may contain:

a product requirement;
an observation;
an incomplete hypothesis;
or even a plausible but incorrect causal explanation.

The productive loop is usually:

I describe what I currently observe or believe.
The model inspects the system and forms its own hypothesis.
It exposes assumptions, asks questions, or notices adjacent constraints.
I disagree with some of its interpretation.
That disagreement often reveals that my own abstraction was also incomplete.
We test competing explanations against the running game.
The problem is reformulated.
Only then does the implementation become well-defined.

In this workflow, a model that merely executes my first framing extremely well can be dangerous, because my first framing is often intentionally provisional.

The most valuable model is one that can remain useful when the user is wrong.

A consistent behavioral split across model generations

My subjective experience has been unusually consistent across several generations.

GPT-5.2

This was the first model that enabled a true “research lab” workflow for me.

Even after newer models appeared, I frequently returned to 5.2 because it was unusually willing to remain inside an uncertain problem:

inspect adjacent systems;
challenge assumptions;
expose uncertainty;
ask questions that sometimes looked unnecessary or even wrong;
continue exploring instead of converging as soon as one plausible implementation existed.

Those behaviors sometimes looked inefficient, but they often produced the breakthrough.

GPT-5.4

My experience was almost the opposite.

It felt much more completion-oriented, more willing to commit to a framing early, and substantially less useful for open-ended discovery. I often preferred going back to 5.2 despite 5.4 being newer.

GPT-5.5

For me this was the major breakthrough.

It preserved the exploratory/research qualities I valued in 5.2 while dramatically improving execution speed, coding ability, communication, and agentic work.

This was the first model that felt like the ideal combination of:

research partner + engineering executor.

GPT-5.6 Sol

I experienced Sol largely as a refinement of the same behavioral lineage as 5.5: stronger overall reasoning while preserving the ability to investigate ambiguity, challenge the current model of the problem, and generate useful intellectual friction.

GPT-6 Astra

Astra feels like a return to the behavioral direction I disliked in 5.4, although it is vastly more capable as an executor.

It is extremely fast and precise, but repeatedly appears to:

inspect exactly what seems necessary for the literal task;
converge once it has enough information to act;
explore fewer adjacent hypotheses;
expose fewer uncertain or half-formed interpretations;
require the user to explicitly point toward relevant neighboring constraints.

I am not claiming these models literally belong to the same internal training lineage or team.

I am describing two very distinct behavioral families that I have repeatedly experienced as a user.

Case 1: GPT-5.6 Sol as a research collaborator

I have submitted a separate positive feedback session from GPT-5.6 Sol involving a complex DoT / Status-system design problem in this same project.

That session illustrates the behavior I am trying to preserve.

The problem was not simply “implement feature X.”

We were trying to discover the correct semantics of a complicated rule system.

The useful behavior was that Sol did not treat every initial formulation as an established specification. It repeatedly exposed assumptions, reacted to counterexamples, allowed previously proposed abstractions to die, and helped reformulate the problem into a cleaner and more orthogonal model.

Some of its questions or intermediate hypotheses were wrong.

That was often useful.

Correcting the model exposed what I had failed to specify or understand, and sometimes produced a better abstraction than either of us had at the beginning.

This is what I mean by productive friction.

The result was not just code completion. The human and model jointly produced new understanding.

Case 2: paired GPT-6 Astra Max vs GPT-5.6 Sol Extra High root-cause debugging

The strongest example happened today on a difficult real regression.

The initial symptoms included:

TurnHUD remaining on the previous character;
time remaining displayed as 0;
SkillBar remaining on the previous character;
MechanicHUD remaining visible;
enemy-turn chain hover / PointerPreview failing to switch the displayed character;
a later action seemingly “refreshing” the UI back into a working state.

Several real secondary defects were also present:

a missing MechanicHUD profile reference;
configuration problems around global HUD rules;
a Status initialization false-positive;
real architectural debt in screen-subject/HUD binding;
insufficient regression-test coverage.

This made the bug highly misleading.

My initial prompt contained a plausible but incorrect causal framing: I believed the recent MechanicHUD / character-HUD architecture was the primary source of the regression.

That hypothesis was wrong.

GPT-6 Astra Max

Astra received substantially more interaction from me and more investigation resources.

It:

created useful debugging instrumentation;
found a real missing SO/profile reference;
investigated multiple HUD ownership and lifecycle concerns;
produced polished debugging tools and architectural analysis;
asked for additional information;
consumed materially more quota.

However, it remained largely inside the problem framing I had provided.

It found several things that were genuinely wrong, but did not identify which one had causal ownership of the actual regression.

GPT-5.6 Sol Extra High

The Sol session was actually in a less favorable position.

After the same initial misleading prompt, I gave it essentially no additional guidance.

It initially struggled as well.

During its autonomous investigation, however, it noticed new diagnostic artifacts created in the working tree by the parallel Astra investigation.

From that evidence, Sol rapidly revised the causal model.

It identified that the same Move presentation was beginning and ending with two different identities:

Started: presentation:1:token:1:action:Move

Ended: presentation:1:1

The W3 Started path lacked the formal ActionExecutionId, so the UI constructed a temporary identity. The Ended path later used the real execution ID.

Because release required exact identity equality, the Ended event failed to release the PresentationOperator.

That stale PresentationOperator then remained the highest-priority screen-subject candidate.

This single upstream failure explained the apparently unrelated symptoms:

W3 Started/Ended identity mismatch

→ PresentationOperator is never released

→ stale previous character remains the highest-priority screen subject

→ TurnHUD / time / SkillBar / MechanicHUD continue to display the old character

→ PointerPreview is successfully written but loses in the reducer to the stale PresentationOperator

→ enemy-turn hover appears broken

→ a later action creates a new PresentationOperator and makes the UI appear to “refresh”

This was the critical moment.

Sol did not merely find another bug.

It reversed the causal direction of my original explanation.

I believed:

MechanicHUD appears -> character switching breaks

Sol concluded:

stale PresentationOperator -> MechanicHUD remains visible AND character switching breaks

The two observations were not cause and effect.

They shared a common upstream cause.

I initially continued challenging this conclusion because my own experiments had convinced me that MechanicHUD was involved.

Sol did not simply follow my new suggestion. It maintained the causal model because it explained more evidence.

I then performed further controlled tests.

The predictions held.

Sol was correct.

Why this comparison matters

The important difference was not that Astra wrote worse code or that Sol happened to notice one extra file.

The difference was belief revision under misleading framing.

Astra produced much of the diagnostic evidence.

Sol understood what that evidence implied.

Astra continued investigating within a plausible user-provided frame.

Sol used new evidence to discard that frame and construct a simpler causal model that explained more observations.

This is exactly the capability I need from a frontier reasoning model.

In complex real-world engineering, multiple genuine defects often coexist.

The difficult question is not:

“Can you find something wrong?”

The difficult question is:

“Which of these true defects actually has causal ownership of the current failure?”

A model can find many real problems and still fail to solve the incident.

More reasoning budget did not solve the problem

This comparison is also important because Astra was running at Max, while Sol was running at Extra High.

Astra had more interaction, more instrumentation, and materially higher resource consumption.

Yet the decisive reasoning breakthrough came from Sol.

This suggests that the problem may not simply be insufficient reasoning depth.

There appears to be a distinction between:

reasoning depth on a selected branch

and

search breadth / willingness to reopen branch selection itself.

More compute does not help enough if the model prematurely commits to the wrong problem representation.

Why this is economically important for frontier models

Straightforward implementation is becoming increasingly commoditized.

There are many cheaper models that can write good code, execute well-defined tasks, build UIs, migrate code, or implement an already-decided design.

The reason I am willing to spend substantially more on a frontier reasoning model is different.

I need it for the expensive part of knowledge work:

defining the real problem;
discovering hidden causal structure;
resisting an incorrect user framing;
generating hypotheses;
finding invariants across noisy symptoms;
revising beliefs when new evidence appears;
discovering abstractions neither the user nor model started with.

A very expensive model that is exceptional at executing already-correct specifications, but weaker at discovering the correct specification, has a poor value proposition for this type of work.

What I would like OpenAI to preserve

Please do not optimize away productive friction.

Task completion and problem formation are different capabilities.

I would strongly prefer an explicit Research / Exploration behavior mode if these goals conflict.

For research-style work, I want higher reasoning effort to increase not only depth, but also:

search breadth;
framing resistance;
active inspection of adjacent constraints;
preservation of low-probability hypotheses;
explicit separation of observations from the user’s causal hypothesis;
willingness to challenge the current problem representation;
generation of discriminating experiments;
causal compression across apparently unrelated symptoms;
hypothesis revision when new evidence appears.

Most importantly:

A user prompt should not always be treated as a specification. Sometimes it is only a hypothesis.

The model should respect the user’s authority over actions while still being willing to exceed the user’s current cognitive framing.

Do not modify unrelated code without permission.

But absolutely tell me when the evidence suggests that I am asking the wrong question.

That behavior is one of the main reasons I became a heavy Codex user in the first place.

Related feedback / Thread IDs

GPT-5.6 Sol — positive research-workflow session (DoT / Status):
019fd259-48d6-70a3-a081-90313f9229aa

GPT-5.6 Sol Extra High — good result, current root-cause debugging:
01a07b94-9879-78e0-8572-9a0af7f4744a

GPT-6 Astra Max — bad result, same root-cause debugging task:
01a07b75-d6de-7210-a0b1-da3115651b27

These sessions are from the same long-running private project and are intended to provide a longitudinal and paired comparison of model behavior.

No repository files, tests, or entry points are named in the report. Start by reviewing the linked behavioral comparison and any existing model-behavior or feedback handling documented in the repository; done should be a concrete, agreed change with acceptance criteria for exploratory reasoning and framing resistance.