openai / openai/codex

Context compaction rewrites the stored rollout in place and permanently destroys the conversation transcript

Open
#44,363 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

app app-server bug context session
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

What version of the Codex App are you using (From “About Codex” dialog)?

Version 26.903.61454

What subscription do you have?

Pro

What platform is your computer?

Darwin 25.6.0 arm64 arm

What issue are you seeing?

Environment

  • Codex desktop (ChatGPT.app) 26.903.61454 (build 8378), macOS 26.6.1, Apple Silicon
  • Session cli_version recorded in rollout: 0.144.0-alpha.4
  • Long-running remote-control session, hosted on a Mac mini, accessed from iPhone / another Mac

Summary

Context compaction rewrote the session's rollout .jsonl in place, replacing the durable
transcript with a compacted summary. The file went from 851,046,757 bytes / 122,877 records
to 7,117,868 bytes / 762 records. All 3,777 assistant messages were discarded. Only the
584 user messages survived, embedded in a replacement_history array inside a single
compacted record, alongside one encrypted_content compaction entry.

Because the app renders thread history from the rollout, the entire visible conversation
disappeared from every client simultaneously (Mac mini, MacBook, iPhone). The session itself kept
running and kept responding.

Compaction should bound what the model carries in context. It should not be destructive to the
stored record. Here the working context and the durable transcript are the same file, so trimming
one destroyed the other.

Impact

  • Two months of conversation (2026-07-11 → 2026-09-09) became unrenderable and unrecoverable
    from the live installation.
  • Loss is silent. No warning before, no error after. The thread still lists and still accepts
    messages; it just shows nothing.
  • Recovery was only possible because an APFS local Time Machine snapshot happened to exist from
    12 minutes before the rewrite.

Timeline (local time)

Time Event
2026-07-11 15:27:42 Session starts. Rollout created.
~2 months Normal use. 175 compacted records accumulate; the file retains full history throughout.
2026-09-09 17:59:21 Rollout file recreated (birth timestamp), 851 MB → 7.1 MB. Assistant turns discarded.
2026-09-09 18:50:50 legacy_to_paginated_v1 migration runs against the now-truncated file.
2026-09-09 18:54 User notices blank history on all three clients.

Note the filename still encodes the original July start time
(rollout-2026-07-11T15-27-42-<thread-id>.jsonl) while the inode's birth time is the compaction
moment. Records at the head of the rewritten file carry July timestamps but contain September
message IDs, which is what makes the rewrite visible.

Reproduction

Not deliberately reproduced — this was a production session. Conditions present:

  1. Very long-lived thread (2 months, ~3.07 billion cumulative tokens reported in token_count).
  2. Multi-agent use — the thread spawned ~170 sub-agent threads.
  3. Always-loaded remote-control session (auto-resumed via startup_prewarm on every app-server start).
  4. Context compaction triggered on a rollout of ~851 MB.

Secondary bug: migration silently skips the compacted rollout

The rewritten rollout carries a ContextCompaction item plus world_state and
token_usage_record record types. legacy_to_paginated_v1:

  • flipped threads.history_mode to paginated,
  • wrote thread_history_projection_state with the cursor parked at EOF
    (next_rollout_byte_offset = file size), so no backfill can ever occur,
  • projected zero items,
  • recorded no row in rollout_migration_skipped_rollouts.

Every other thread on the machine (2,000+) migrated correctly. Re-running the sweep will not
revisit this thread, because history_mode is already paginated.

Third issue: always-loaded threads never read stored history

For a thread that is loaded, thread/resume takes resume_running_thread and composes the
response from the in-memory session, which starts empty on each load. It never consults
thread_items. Combined with startup_prewarm auto-loading this session at app-server startup,
the stored-history path is unreachable for this thread — repopulating thread_items correctly
had no effect on what the client displayed.

Relevant log line, repeating on every open:

resume_running_thread: thread/resume overrides ignored for loaded thread <id>:
config overrides were provided and ignored while running

Also observed

The desktop client requests experimental feature keys the bundled app-server rejects as invalid,
including one directly related to this area:

ignoring invalid experimental feature enablement keys:
apps_mcp_path_override, background_paginated_rollout_migration, local_thread_store_compression

Expected behavior

  1. Compaction must not mutate or truncate the stored rollout. Bound the model's context window
    without destroying the durable record — separate files, or append a compaction marker while
    retaining prior records.
  2. If a destructive rewrite is genuinely intended, retain the pre-compaction file and warn.
  3. A migration that projects zero items must record a skip with a reason rather than marking the
    thread migrated.
  4. A loaded thread should fall back to stored history when its in-memory item list is empty.

Artifacts available

  • Pre-compaction rollout, 851,046,757 bytes, recovered from an APFS snapshot (SHA-256 available).
  • Post-compaction rollout, 7,117,868 bytes.
  • App-server logs covering the migration and every resume.
What steps can reproduce the bug?

Use the app. Watch your thread get destroyed.

What is the expected behavior?

No response

Additional information

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the rollout handling around context compaction and the named entry points legacy_to_paginated_v1, resume_running_thread, and startup_prewarm. Compare the recovered pre-compaction rollout, the rewritten rollout, and the app-server logs. Done means durable history remains available after compaction, zero-item migrations are not falsely marked complete, and loaded threads can display stored history when their in-memory list is empty.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.