openai / openai/codex

Perceived quality regression in desktop voice: repeated inaccurate diagrams and failure to follow corrections

Open
#44,428 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

app bug imagen model-behavior
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

What version of the Codex App are you using?

Not captured. This report concerns a macOS Codex desktop voice session with a backend coding agent and built-in image generation.

What subscription do you have?

Not captured.

What platform is your computer?

macOS.

What issue are you seeing?

I experienced a substantial perceived drop in reasoning, instruction-following, and explanation quality during a long-running voice-assisted technical task. The assistant repeatedly produced verbose, confusing explanations and inaccurate diagrams despite concrete workflow code, stored JSON outputs, screenshots, and explicit corrections being available.

The task was straightforward: explain how a large intermediate JSON result becomes a smaller final table, showing what each workflow step receives, retains, and displays, for one specific example.

Observed failures in this session:

  • A generated diagram grouped only two supporting sections under a retention bracket when the explanation applied to four. The user had to identify the error.
  • The explanation blurred the distinction between the full object passed into a step and the subset of that object actually read by the step.
  • A replacement image introduced a misleading connector; it required another edit to remove it.
  • Repeated diagrams required readers to move right, then restart from the left below, instead of following the explicitly requested simple aligned layout.
  • The assistant repeatedly mixed the immediate explanation with tangential readiness caveats and row counts, making a simple question harder to understand.
  • The assistant acknowledged that it had not reviewed earlier images carefully enough before presenting them.

The user suspects a model downgrade or routing change because the quality felt substantially worse than expected from Astra. This is a user hypothesis, not a verified diagnosis. There is no established evidence here of fallback to GPT-4 or any other particular model.

What steps can reproduce the bug?

This is an observed session-quality failure; a deterministic minimal reproduction has not been established.

A privacy-safe approximation of the task:

  1. Use desktop voice to discuss a workflow whose SDK returns five top-level sections: business data, extracted records, reconciliation, flags, and a signature.
  2. Supply code showing that a validator retains all five; a diagnostic step receives the full object but displays selected business fields; separate payload steps consume the business section and feed the final view.
  3. Supply screenshots of both the large intermediate result and the smaller final table.
  4. Ask for a clear, image-generated comparison of each step's input, action, and output for one example, with consistent blocks and colors.
  5. Explicitly correct inaccurate brackets, confusing branch directions, and confusion between step inputs and displayed outputs.
  6. Observe whether subsequent explanations and images honor the corrections without introducing new misleading relationships.

What is the expected behavior?

A concise explanation grounded in the inspected code and outputs; a readable diagram or aligned table that preserves actual data dependencies; clear distinction between retained data and displayed fields; and visual verification before presenting a generated image. Corrections should carry forward across voice handoffs and revisions.

Additional information

Please investigate instruction-following and visual-review quality across the desktop voice/backend-agent/image-generation interaction. If internal diagnostics are available, check whether model routing, reasoning settings, or long-conversation context handling contributed. Do not treat the suspected model downgrade as established fact.

This report was prepared and submitted by Codex at the user's explicit request. Customer data, private repository contents, screenshots, and the full conversation are intentionally excluded from this public issue.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files or tests are named. Start at the desktop voice/backend-agent/image-generation interaction and try the privacy-safe approximation described in the issue, comparing outputs before and after explicit corrections. Done means establishing a reproducible failure or identifying whether routing, reasoning settings, or long-conversation context handling contributed.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos, rust
Domain
ai, audio-video-rtc, desktop
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.