Subagent spawns duplicate the full parent session history into each child rollout file (quadratic disk growth)

Open
#36,557 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Quiet
Tech stack
rust
Domain
cli, performance

Research direction

Start with the rollout files under ~/.codex/sessions/ and reproduce the reported growth by spawning subagents and triggering compaction repeatedly. Trace how child rollouts receive parent history and how replacement_history is written. Done means avoiding full-history duplication and repeated compaction rewrites without losing session history.

Written by the indexing model from the issue text.

Description

bug CLI performance session subagent

Summary

When a session spawns subagents, each child's rollout .jsonl under ~/.codex/sessions/ begins with a complete copy of the parent session's history. Compaction events additionally re-write the entire replacement_history into the same log. For a long-lived session that spawns many subagents, disk usage grows roughly quadratically with session length.

Impact (real numbers from one machine)

  • One long-running thread (~3 weeks old, ~100 subagent spawns, ~1,400 compaction events) produced 37 rollout files over 800MB each in a single day — ~30GB/day, measured at ~97% duplicated bytes.
  • ~/.codex/sessions/ reached 42GB, and the 1TB disk hit 3.8GB free before the pattern was found.
  • Session profiling: 93 of 101 rollout files created that day traced to the same parent thread.

Repro

  1. Start a session; build up a few hundred KB of history.
  2. Spawn several subagents (any task).
  3. Observe each new rollout-*.jsonl starts with the parent's full serialized history; sizes grow with parent history length, not child work.
  4. Trigger compaction repeatedly; observe replacement_history re-written in full each time.

Suggested fixes

  • Reference the parent rollout by ID + offset instead of copying history into children.
  • Store compaction replacement_history as a delta or in a separate file, not appended in full to the live log.
  • Alternatively: transparent zstd compression of rollout files (we measured 10–20× on this content).

Environment

  • Codex CLI with Codex Desktop, macOS (Darwin 25.5), model gpt-5.6-terra, subagents used heavily via the native multi-agent feature.
Dominant language
Rust
Stars
125k
Forks
19.5k
Avg merge
1m
Merged PRs (30d)
1k

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from openai/codex

All issues in openai/codex

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.