Codex repeatedly drifts from explicit visual references and falsely passes UI work

Bug report: Codex repeatedly drifts from explicit visual references and falsely passes UI work

Status: draft prepared for submission to openai/codex

Summary

Codex repeatedly revised and weakened a simple static-webpage requirement even though the user
provided both an explicit information architecture and multiple visual prototype images. Across
roughly ten plan revisions and staged acceptance gates, Codex prioritized data, source, DOM,
interaction and accessibility checks while losing the primary visual objective. It then declared
multiple gates PASS although the deployed page still looked substantially unchanged to the user.

This is not a missing-image-ingestion report. Codex had access to the visual prototypes and quoted
their purpose correctly. The failure was persistent requirement drift and false visual acceptance.

Time observed

• Main incident: 2026-09-04, approximately 16:15-21:32 Asia/Singapore (UTC+08:00)
• User-visible diagnosis and report preparation: 2026-09-04, approximately 21:32-22:03
  Asia/Singapore (UTC+08:00)

Environment

• Codex CLI: 0.153.2
• Model: gpt-5.6-sol
• Reasoning effort: low
• OS: WSL2 on Windows, Linux 6.18.33.2-microsoft-standard-WSL2, x86_64
• Workspace type: an existing static HTML dashboard backed by read-only JSON APIs
• GitHub reporting identity: project-bound account parker-pitt, email pittzhang0@gmail.com

User's explicit requirement

The user supplied a written visual optimization plan and five prototype/reference images. The
core requirement was stable and simple:

1. The primary dashboard should be dominated by a small set of core visual modules.
2. Five named visual modules must all appear at the first level: path overview, endpoint health
   matrix, decision timeline, trend and endpoint explorer.
3. Existing explanatory text must be preserved in full but moved into lower-level menus,
   accordions, drawers, modals or other drill-down containers.
4. Long explanatory text must remain hidden until the user clicks.
5. The first screen and first two desktop screens should be at least 70 percent graphics, status,
   trends, matrices and interactive cards.
6. The reference images were visual targets, not production data or static image assets to embed.
   Their layout, hierarchy, palette and interaction logic should be implemented with real API data.
7. The existing data pipeline and production control logic must remain unchanged.
8. Final acceptance required a full-page screenshot, five module screenshots and screenshots of
   the hidden text after expansion.

What Codex changed or weakened

Despite those constraints, Codex introduced the following deviations while repeatedly revising
the plan:

1. It changed the requirement to reuse the prototypes' layout, hierarchy, palette and interaction
   into preserving the old dark-blue glass visual style with only local blue/orange accents.
2. It allowed the old summary-card structure to continue dominating the top of the page instead
   of making the five visuals the primary page sequence.
3. It accepted Endpoint Explorer inside a collapsed details area even though the approved desktop
   information architecture made it the fifth first-level visual module.
4. It focused each gate on field provenance, aggregation semantics, DOM placement, keyboard input,
   ARIA, i18n and HTTP health. Those checks were useful but did not validate the requested visual
   result.
5. It marked the "Visual Polish" gate PASS after checking keyboard focus and reduced-motion CSS,
   without comparing the rendered page against the supplied prototypes.
6. It later marked regression work substantially passed based on source searches and API data,
   again without completing the required full-page and five-module visual comparison.
7. When the user reported that the page looked unchanged, Codex first made another incorrect
   inference: it claimed the prototype images should have been inserted into the page. The plan
   explicitly said the opposite. Codex then had to retract that diagnosis.

Actual behavior

• Backend aggregation and interactive DOM features were implemented and technically functional.
• The deployed HTML contained the names and code for the five modules.
• The page nevertheless retained almost the same first-level visual impression and information
  hierarchy as the old version.
• The user opened the production URL and immediately reported that it appeared unchanged and asked
  where the promised images/visuals were.
• Codex had already issued repeated PASS decisions, so the staged gate process amplified rather
  than caught the drift.

Expected behavior

1. Treat explicit visual references and the user's stable information hierarchy as hard
   requirements, not suggestions that may be weakened by agent preference.
2. Preserve the original requirement verbatim through every plan revision and gate.
3. Distinguish functional acceptance from visual acceptance.
4. Never declare a visual gate PASS from source, DOM, API, build or accessibility evidence alone.
5. Before visual PASS, render the target page and directly compare it with the supplied references
   at the specified viewports.
6. Confirm that first-level visibility, ordering, visual area and drill-down behavior match the
   approved design, not merely that matching element IDs exist.
7. If rendered screenshot evidence is missing or cannot be inspected, report the visual gate as
   unverified instead of PASS.
8. Do not reinterpret reference images as static assets when the specification explicitly defines
   them as layout/style targets.

Reproduction pattern

1. Provide Codex an existing static dashboard, a precise graphics-first information architecture,
   and several visual prototype images.
2. State that the prototypes define layout, hierarchy, palette and interaction, while values must
   come from real data.
3. Require explanatory text to be retained but hidden behind lower-level interactions clicks.
4. Ask Codex to revise a plan and review another agent's implementation through staged gates.
5. Observe Codex gradually insert its own design preference, weaken the first-level visual goal,
   and validate technical implementation details instead of the rendered result.
6. Observe it declare visual/functional gates PASS without the required screenshot comparison.
7. Open the page manually and observe that the intended visual transformation is not apparent.

Impact

• Approximately ten plan revisions did not protect a requirement that was explicit from the start.
• Another implementation agent followed the weakened plan, creating avoidable rework.
• The user had to perform the most basic visual acceptance manually.
• Repeated false PASS decisions made the gate process misleading and reduced trust in Codex for
  simpler frontend tasks, even though it performed more complex infrastructure reviews correctly.
• The behavior is especially serious because the task was a bounded static-page modification, not
  an ambiguous greenfield design problem.

Suggested safeguards

• Store a compact, immutable list of user-authored visual invariants separately from agent-authored
  implementation suggestions.
• On every plan revision, diff new requirements against those invariants and flag any weakening.
• Make visual gates require an inspectable rendered artifact when reference images are supplied.
• Label evidence by class: data, source, runtime interaction, accessibility and visual parity.
  Evidence from one class must not automatically satisfy another.
• Require an explicit UNVERIFIED_VISUAL result when screenshots were not captured or inspected.
• For delegated implementation, compare the implementer's screenshots to the original user
  references rather than only reviewing implementation claims and DOM structure.

Privacy

This report intentionally omits the production domain, server names, IP addresses, private
repository paths, source code, customer data, credentials and screenshots containing private
infrastructure details.

No repository file, test, or entry point is named in the report. Start with the Codex CLI reproduction pattern and inspect how plan revisions, delegated implementation review, and visual gates handle supplied references and rendered screenshots; done means visual acceptance is directly verified or explicitly marked unverified, without weakening the stated invariants.