[macOS][0.153.4] Paginated history stalls at duplicate ordinal; controlled recovery comparison and stable backport request

Summary

A local Codex Desktop task on macOS kept displaying early conversation history while the canonical JSONL continued to contain later user/assistant messages. The derived history projection was stuck at a duplicate ordinal.

This report adds a controlled comparison using the same bundled 0.153.4 binary and the same immutable snapshot: reopening did not recover it, rebuilding the derived rows failed again, and a narrowly guarded checkpoint correction recovered 24 turns without rewriting the canonical transcript. It also identifies the difference between the stable tag and the recovery handling merged in #42369.

Related: #43124, #43182, #43640, #41566, #41079. Please consolidate if this belongs to the same defect; the additional evidence here is the three-way recovery comparison and stable-tag behavior.

Environment

• Desktop bundle: ChatGPT.app, CFBundleIdentifier com.openai.codex
• Bundle version: 26.901.51231, build 8109 (read from Info.plist, not the About dialog)
• Bundled CLI/app-server used for all experiments: codex-cli 0.153.4
• OS: macOS 27.0, Apple Silicon arm64
• Local task; history_mode=paginated
• Observed September 8, 2026

Exact failure boundary, sanitized

The only ordinal discontinuity in the captured complete-line transcript prefix was:

Record	Root ordinal
event_msg/token_count	272
event_msg/thread_settings_applied	272
event_msg/task_started	273

The projection expected ordinal 273 at byte offset 4,813,257, which was exactly the start of the second ordinal-272 record. That metadata record occupied 1,992 bytes. The next complete record began at offset 4,815,249 and had ordinal 273.

Runtime warning, identifiers replaced:

failed to project durable rollout for THREAD:
thread-store internal error:
thread history projection for ROLLOUT expected ordinal 273, got 272

The active rollout has a separate rollout identifier from the logical thread identifier; the stuck projection row is keyed by that rollout identifier. This distinction matters when diagnosing an edited/replaced rollout. The original action that caused ordinal reuse has not been established.

The desktop history reader returned one old turn with no next cursor while later messages remained in the canonical JSONL. Repeated application runs continued to encounter the same boundary. This is history accessibility failure, not evidence of deletion of the observed later messages.

Controlled local comparison

An immutable snapshot contained 26,925,306 bytes in the affected rollout. Separate test CODEX_HOME directories received the same target-thread state and history rows and byte-identical rollout copies. No credentials were copied, no model turn was started, and hooks were disabled in the test configuration.

Each test initialized the installed app-server, read thread/turns/list, called thread/resume with excludeTurns=true, and read thread/turns/list again.

Test	Result
Existing checkpoint, reopen/resume	Still 1 old turn; same expected-273/got-272 warning
Remove only this rollout's derived projection/turn/item rows in the test copy, then resume	0 projected turns; rebuild fails again at the same duplicate ordinal
Advance only this rollout's next_rollout_byte_offset past the verified duplicate metadata record, retaining expected ordinal 273	24 turns and 1,560 items recovered; no ordinal warning

The recovered snapshot included 26 userMessage and 123 agentMessage items. SQLite integrity_check returned ok. The copied canonical transcript's SHA-256 remained unchanged. After restarting the isolated app-server again, all 24 turns remained available and thread/items/list returned populated pages.

The migration CLI was also checked without --apply:

codex migrate-rollouts --thread <redacted> --json
status: already_paginated
bytes_processed: 0

It does not provide a repair path for this already-paginated thread.

Recovery on the affected installation

After backing up both state/history databases and the transcript prefix, the same narrowly scoped cursor correction was applied to one projection row, using BEGIN IMMEDIATE and old-value predicates. The expected ordinal was not changed, no history row was deleted, and no canonical record was rewritten. The existing trigger on this table is AFTER DELETE only; the operation was UPDATE.

The native live writer subsequently caught up. Read-only database checks found 24 projected turns, the checkpoint resumed advancing with new output, and the app-provided newest-first history reader returned recent turns with hasMore=true instead of the stale exhausted page. The pre-repair transcript prefix remained hash-identical. Desktop visual confirmation is tracked separately; a history API response alone is not claimed as screenshot evidence.

This is diagnostic recovery evidence for one verified metadata boundary, not a recommendation to skip arbitrary duplicate records or user/assistant content.

Stable source versus upstream fix

• The public rust-v0.153.4 materializer still returns an error for an ordinal lower than the expected ordinal. The observed binary behavior agrees.
• PR #42369, merged September 2, changes the mainline materializer to move past duplicate/regressed ordinals and other invalid records.
• As of this investigation, the GitHub latest stable release is rust-v0.153.4. A mainline fix being merged does not establish that affected stable desktop installations contain it.

Please clarify the first desktop/stable runtime that includes this recovery handling and whether already-wedged checkpoints recover automatically after upgrading.

Expected fix

1. Ship/backport the applicable safe projector recovery to the stable desktop runtime.
2. Prevent ordinal reuse at writer/resume boundaries.
3. Offer a supported per-thread history integrity/recovery command that handles already_paginated sessions.
4. Show an actionable incomplete-history state when durable records remain beyond a failed checkpoint, rather than silently presenting old history as complete.
5. Cover this token_count(N) -> thread_settings_applied(N) -> task_started(N+1) boundary, resumed-history pagination, populated message pages, and restart persistence in regression tests.

No raw transcripts, databases, prompts, credentials, private identifiers, filesystem paths, internal URLs, or full logs are attached. The writer-side initiating trigger and future recurrence on 0.153.4 remain unproven.

Start with codex-rs/thread-store/src/local/thread_history_materialization.rs and compare the stable rust-v0.153.4 behavior with the recovery handling in PR #42369. Check the migration CLI behavior for already_paginated sessions and identify the applicable stable runtime path. Done should include regression coverage for the duplicate-ordinal boundary, resumed pagination, populated pages, and restart persistence.