Context compaction rewrites the stored rollout in place and permanently destroys the conversation transcript
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 125k
- Forks
- 19.4k
- PR merge metrics
- PR metrics pending
Description
What version of the Codex App are you using (From “About Codex” dialog)?
Version 26.903.61454
What subscription do you have?
Pro
What platform is your computer?
Darwin 25.6.0 arm64 arm
What issue are you seeing?
Environment
- Codex desktop (ChatGPT.app)
26.903.61454(build 8378), macOS26.6.1, Apple Silicon - Session
cli_versionrecorded in rollout:0.144.0-alpha.4 - Long-running remote-control session, hosted on a Mac mini, accessed from iPhone / another Mac
Summary
Context compaction rewrote the session's rollout .jsonl in place, replacing the durable
transcript with a compacted summary. The file went from 851,046,757 bytes / 122,877 records
to 7,117,868 bytes / 762 records. All 3,777 assistant messages were discarded. Only the
584 user messages survived, embedded in a replacement_history array inside a single
compacted record, alongside one encrypted_content compaction entry.
Because the app renders thread history from the rollout, the entire visible conversation
disappeared from every client simultaneously (Mac mini, MacBook, iPhone). The session itself kept
running and kept responding.
Compaction should bound what the model carries in context. It should not be destructive to the
stored record. Here the working context and the durable transcript are the same file, so trimming
one destroyed the other.
Impact
- Two months of conversation (2026-07-11 → 2026-09-09) became unrenderable and unrecoverable
from the live installation. - Loss is silent. No warning before, no error after. The thread still lists and still accepts
messages; it just shows nothing. - Recovery was only possible because an APFS local Time Machine snapshot happened to exist from
12 minutes before the rewrite.
Timeline (local time)
| Time | Event |
|---|---|
| 2026-07-11 15:27:42 | Session starts. Rollout created. |
| ~2 months | Normal use. 175 compacted records accumulate; the file retains full history throughout. |
| 2026-09-09 17:59:21 | Rollout file recreated (birth timestamp), 851 MB → 7.1 MB. Assistant turns discarded. |
| 2026-09-09 18:50:50 | legacy_to_paginated_v1 migration runs against the now-truncated file. |
| 2026-09-09 18:54 | User notices blank history on all three clients. |
Note the filename still encodes the original July start time
(rollout-2026-07-11T15-27-42-<thread-id>.jsonl) while the inode's birth time is the compaction
moment. Records at the head of the rewritten file carry July timestamps but contain September
message IDs, which is what makes the rewrite visible.
Reproduction
Not deliberately reproduced — this was a production session. Conditions present:
- Very long-lived thread (2 months, ~3.07 billion cumulative tokens reported in
token_count). - Multi-agent use — the thread spawned ~170 sub-agent threads.
- Always-loaded remote-control session (auto-resumed via
startup_prewarmon every app-server start). - Context compaction triggered on a rollout of ~851 MB.
Secondary bug: migration silently skips the compacted rollout
The rewritten rollout carries a ContextCompaction item plus world_state and
token_usage_record record types. legacy_to_paginated_v1:
- flipped
threads.history_modetopaginated, - wrote
thread_history_projection_statewith the cursor parked at EOF
(next_rollout_byte_offset= file size), so no backfill can ever occur, - projected zero items,
- recorded no row in
rollout_migration_skipped_rollouts.
Every other thread on the machine (2,000+) migrated correctly. Re-running the sweep will not
revisit this thread, because history_mode is already paginated.
Third issue: always-loaded threads never read stored history
For a thread that is loaded, thread/resume takes resume_running_thread and composes the
response from the in-memory session, which starts empty on each load. It never consults
thread_items. Combined with startup_prewarm auto-loading this session at app-server startup,
the stored-history path is unreachable for this thread — repopulating thread_items correctly
had no effect on what the client displayed.
Relevant log line, repeating on every open:
resume_running_thread: thread/resume overrides ignored for loaded thread <id>:
config overrides were provided and ignored while running
Also observed
The desktop client requests experimental feature keys the bundled app-server rejects as invalid,
including one directly related to this area:
ignoring invalid experimental feature enablement keys:
apps_mcp_path_override, background_paginated_rollout_migration, local_thread_store_compression
Expected behavior
- Compaction must not mutate or truncate the stored rollout. Bound the model's context window
without destroying the durable record — separate files, or append a compaction marker while
retaining prior records. - If a destructive rewrite is genuinely intended, retain the pre-compaction file and warn.
- A migration that projects zero items must record a skip with a reason rather than marking the
thread migrated. - A loaded thread should fall back to stored history when its in-memory item list is empty.
Artifacts available
- Pre-compaction rollout, 851,046,757 bytes, recovered from an APFS snapshot (SHA-256 available).
- Post-compaction rollout, 7,117,868 bytes.
- App-server logs covering the migration and every resume.
What steps can reproduce the bug?
Use the app. Watch your thread get destroyed.
What is the expected behavior?
No response
Additional information
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the rollout handling around context compaction and the named entry points legacy_to_paginated_v1, resume_running_thread, and startup_prewarm. Compare the recovered pre-compaction rollout, the rewritten rollout, and the app-server logs. Done means durable history remains available after compaction, zero-item migrations are not falsely marked complete, and loaded threads can display stored history when their in-memory list is empty.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100