[Data Loss] Background legacy-to-paginated migration can replace persistent subagent history with bounded resume context without preserving the original

Summary

A background legacy-to-paginated rollout migration appears capable of causing silent loss of persistent task history.

In one real persistent, user-facing Codex task, 29 task-owned assistant replies from 10 separate turns are present in multiple independently preserved pre-migration snapshots but are completely absent from the current canonical JSONL.

The missing replies contain 41,278 normalized text characters.

This is not inferred from file size or record count. The complete reply bodies were searched across every string field in the current JSONL, including response records, events, and compaction replacement history, and are no longer present.

The current JSONL is still syntactically valid and the task remains resumable. This makes the failure particularly dangerous: semantic history can be lost while the task continues to appear healthy.

This appears closely related to:

• #45500 — persistent app-created tasks losing original transcripts on bounded subagent history migration
• #38762 — persisted subagent threads ending with empty projected history after migrate-rollouts
• #46485 — older conversation hidden after reopening while messages remain in JSONL

This report adds a verified pre-migration snapshot comparison proving that complete task-owned assistant replies are absent from the current canonical JSONL.

Impact

Confirmed in the affected task:

• 29 missing assistant response records
• 10 task-local turns affected
• 41,278 normalized text characters missing
• the missing replies survive intact in three independently stored, byte-identical cold snapshots
• the lost replies belong to the task's own work, not merely duplicated inherited parent context
• no built-in backup of the original rollout was found in the migration publication path

Without external snapshots, recovery of these verified replies is not established.

The task is persistent and user-facing, despite being stored with:

source=vscode
thread_source=subagent

Before the rewrite:

history_mode=legacy

After the rewrite:

history_mode=paginated

The current rollout also contains a subagent history boundary and the output structure is consistent with the bounded-context migration path described below.

Expected behavior

A storage-format migration must preserve the complete durable task history, even if the model only needs a smaller bounded context in order to resume.

Model resume context and durable human-visible transcript are different data-retention requirements.

At minimum, before replacing the canonical rollout, the migration should either:

1. preserve the complete original transcript,
2. retain an automatic recoverable backup,
3. require explicit confirmation before destructive transformation,
4. or guarantee a lossless rollback path.

A bounded model context should never become the only remaining durable representation of a persistent task's historical transcript.

Actual behavior

For tasks classified as subagents, the migration code selects bounded context sufficient for model resume rather than preserving the complete historical rollout.

That bounded selection is then written into a newly staged rollout.

The staged rollout is subsequently published over the canonical rollout path.

The inspected publication path does not preserve a copy of the original rollout before replacement.

The transaction is atomic in the filesystem sense, but atomic replacement does not guarantee semantic completeness of the history being published.

Relevant source code

The observations match the public source for rust-v0.153.4.

Feature definition:
https://github.com/openai/codex/blob/3d2ee51ca2d5db578f328aa75e20aa22c0197c9a/codex-rs/features/src/lib.rs#L1069-L1074

Background/startup migration entry points:
https://github.com/openai/codex/blob/3d2ee51ca2d5db578f328aa75e20aa22c0197c9a/codex-rs/core/src/thread_manager.rs#L360-L390

Runtime enablement path:
https://github.com/openai/codex/blob/3d2ee51ca2d5db578f328aa75e20aa22c0197c9a/codex-rs/app-server/src/request_processors/config_processor.rs#L256-L300

Subagent classification:
https://github.com/openai/codex/blob/3d2ee51ca2d5db578f328aa75e20aa22c0197c9a/codex-rs/thread-store/src/local/rollout_migration.rs#L430-L445

Bounded-context selection:
https://github.com/openai/codex/blob/3d2ee51ca2d5db578f328aa75e20aa22c0197c9a/codex-rs/thread-store/src/local/rollout_migration/subagent.rs

Replacement rollout writer:
https://github.com/openai/codex/blob/3d2ee51ca2d5db578f328aa75e20aa22c0197c9a/codex-rs/thread-store/src/local/rollout_migration.rs#L835-L874

Boundary rewrite:
https://github.com/openai/codex/blob/3d2ee51ca2d5db578f328aa75e20aa22c0197c9a/codex-rs/thread-store/src/local/rollout_migration.rs#L678-L738

Publication over the canonical rollout:
https://github.com/openai/codex/blob/3d2ee51ca2d5db578f328aa75e20aa22c0197c9a/codex-rs/thread-store/src/local/rollout_migration.rs#L747-L785

Pending-marker implementation:
https://github.com/openai/codex/blob/3d2ee51ca2d5db578f328aa75e20aa22c0197c9a/codex-rs/thread-store/src/local/rollout_migration/publish.rs#L212-L234

The pending marker is transaction metadata, not a retained copy of the original transcript.

Evidence from the affected task

A before/after structural comparison shows:

Measurement	Preserved pre-migration copy	Current canonical JSONL
Top-level records	1,143	635
Direct user-role messages	17	7
Direct assistant messages	35	17
Direct developer messages	8	3
Assistant text characters	59,059	28,906
Top-level compaction records	2	1

These record-count differences alone are not being used as proof of loss because canonicalization legitimately changes representation.

The data-loss finding is instead based on direct text comparison.

The 29 complete assistant replies exist in the preserved copies and are absent from the current canonical JSONL.

The current file also exhibits several fingerprints matching the migration code path:

• legacy → paginated history mode transition
• actual thread_source=subagent metadata
• bounded historical suffix
• subagent history boundary
• collapsed historical outer timestamps
• physical file replacement
• projected history containing zero turns/items despite retained JSONL content

Backup behavior

Three independently stored pre-migration copies preserve all 29 verified missing replies.

No original-rollout backup created by the inspected migration publication path was found.

Temporary migration artifacts and the pending marker are not backups of the source transcript.

Therefore:

DATA_LOSS_IN_CURRENT_TARGET_JSONL = CONFIRMED
RECOVERY_FROM_EXTERNAL_SNAPSHOTS = AVAILABLE
BUILT_IN_ORIGINAL_BACKUP_IN_INSPECTED_PUBLISH_PATH = NOT_IMPLEMENTED
RECOVERY_WITHOUT_EXTERNAL_SNAPSHOTS = NOT_ESTABLISHED

Root-cause confidence

The content loss itself is confirmed.

Attribution to this specific migration code path is currently:

ROOT_CAUSE_CONFIDENCE = HIGH

not CONFIRMED.

Reasons for keeping that distinction:

• no direct per-thread migration-start/publish log event was recovered
• the exact historical effective feature-gate value was not preserved
• the exact historical runtime executable is no longer available for binary hashing

However, the affected task's classification, format transition, bounded history, timestamp rewrite, visibility boundary, filesystem replacement pattern and resulting output are all consistent with the publicly available migration implementation for the runtime version in use.

Environment

Codex Desktop:
26.901.6511.0

Codex runtime:
0.153.4

Public source tag:
rust-v0.153.4

Public source commit:
3d2ee51ca2d5db578f328aa75e20aa22c0197c9a

Platform:
Windows

The exact historical Windows build was not preserved.

Safe reproduction proposal

I have NOT reproduced this destructively against production data.

A safe synthetic reproduction could use a completely disposable Codex home:

1. Create a synthetic legacy rollout with:

   source=vscode
   thread_source=subagent
   

2. Add unique synthetic task-owned user and assistant sentinel messages.

3. Include a valid compaction checkpoint and later task-local turns sufficient for bounded-context selection.

4. Preserve an immutable copy of the original rollout.

5. Run the legacy-to-paginated migration once.

6. Compare whether every task-owned historical sentinel remains durably recoverable in the resulting storage.

7. Separately verify model resumability versus transcript preservation.

Expected invariant:

Limiting context for model resume must not replace or destroy
the only durable archive of the task's own historical transcript.

Controls should include:

• thread_source=user
• subagent without compaction
• genuine inherited child context

Suggested fix / safety requirement

The bounded-context representation used for model resume should not replace the only durable historical transcript.

Possible fixes include:

• preserve the complete original rollout separately from bounded resume context
• create an automatic pre-migration backup before canonical replacement
• keep bounded subagent context as a derived representation rather than the authoritative archive
• require explicit user confirmation before destructive migration
• provide rollback/recovery support
• surface a migration report containing affected tasks and preservation status

A background storage migration should never silently trade transcript preservation for model-resume efficiency.

Privacy

I have a detailed structural evidence package containing hashes, counts, timelines, source-code correlation and missing-record ledgers.

It contains no message bodies or hidden reasoning content.

I am intentionally not posting the affected task identifier, local filesystem paths, workspace name or conversation content publicly.

Start with codex-rs/thread-store/src/local/rollout_migration.rs and its subagent.rs and publish.rs paths, then trace the thread_manager.rs startup entry point through classification, bounded-context selection, boundary rewriting, and publication. Use a disposable synthetic legacy rollout with sentinel messages and an immutable copy; done means migration preserves recoverable task history separately from bounded resume context and demonstrates this with the stated controls.