memory leak: `TurnDiffTracker` grows unboundedly (OOM within hours)

What version of Codex CLI is running?

0.147.0

What subscription do you have?

Pro 5x

Which model were you using?

Gpt-sol-5.6 medium

What platform is your computer?

Linux

What terminal emulator and version are you using (if applicable)?

tmux

Codex doctor report

timing out

What issue are you seeing?

codex 0.147.0 memory leak: TurnDiffTracker grows unboundedly (OOM within hours)

Summary

When the agent edits large text files (tens of MB to GB-scale) via apply_patch,
codex's per-task TurnDiffTracker pins the full baseline and current contents of
every tracked file in RAM, re-renders and re-concatenates the entire accumulated
unified diff into a single String on every patch, and then clones that
whole string to broadcast a TurnDiffEvent — additionally formatting it in full
into the tracing log (debug!("TurnDiffEvent: {unified_diff}")).

There is no size cap anywhere in this pipeline (only a 100 ms time cap on the
diff algorithm — and that path increases memory use by producing whole-file
"+all/-all" replacement hunks). In a long-running session that edits large
generated text files, RSS grows ~10–30 MB/s until the OOM killer terminates the
process (~70 GB observed, limited only by our container memory limit).

Affected versions: at least 0.147.0 (rust-v0.147.0); the relevant code is
identical on main as of 2026-08-18.

Affected binary

• Release binary: codex 0.147.0, target x86_64-unknown-linux-musl
  (statically linked, stripped)
• Size: 258,278,208 bytes
• SHA-256: cb0a15567e9a60a5820d54b0f6ae86d504dc3805c1eab21a47f70e3eb7b73a40
• Note: the binary is stripped (no symbol table) and carries no GNU build ID.

Observed behavior (two consecutive incident processes)

Same long-running automated session, restarted after the first OOM kill:

process	lifetime	memory growth	fate
incident #1	~2h18m	RSS → 70.5 GB (~28 MB/s sustained)	OOM-killed (hit the cgroup memory limit)
incident #2	hours (still running at time of writing)	RSS 13 GB → 16.6 → 28 → 33 → 40 → 45+ GB, sawtoothing upward	leaking

Kernel log for incident #1 (sanitized):

oom-kill:constraint=CONSTRAINT_MEMCG,…,task=codex,pid=NNNN
oom_reaper: reaped process NNNN (codex), now anon-rss:0kB

What the leaked memory contains

• Essentially all of the growth is private anonymous memory (no file backing,
  no shared memory): e.g. 70.7 GB RSS was 70.75 GB anonymous.
• The heap/address space is dominated by one contiguous anonymous VMA that
  cycles through ~10–14 GiB fully-populated mappings (freed and re-mmapped between
  rebuilds; e.g. a 13.62 GiB VMA later replaced by a 12.98 GiB one) plus a handful
  of smaller multi-GB mappings.
• The giant mapping contains, verbatim, concatenated git diff-format text:
  • 183 diff --git a/… b/… sections and 489 hunks in one 12.98 GiB snapshot
  • ~13.1 million + lines and ~13.1 million - lines with essentially
    zero context lines (~2,300 context lines total)
  • hunks are whole-file replacements, e.g. @@ -1,107773 +1,107775 @@
  • headers use the absolute-path fallback form (diff --git a//abs/path b//abs/path), matching TurnDiffTracker::render_diff
• The diffed payloads correspond to the large generated text files (57 MB – 2.3 GB
  each) that the agent session was repeatedly regenerating.
• RSS sawtooths (e.g. 17.3 → 16.7 GB in 15 s) as each rebuilt aggregated diff
  String replaces the previous one, while the cumulative sum of tracked content
  keeps climbing. Net new diff text appears at ~20–30 MB/s while the main thread
  and async workers burn CPU continuously.
• The session's debug-log database grew to ~1.8 GB (db + WAL) with busy SQLite
  writer threads — consistent with the tracing pipeline serializing the entire
  diff on every event (see root cause #4).

Root cause

Verified against the upstream source at tag rust-v0.147.0 (identical logic on
main at the time of writing).

1. Full file contents retained for the whole task —
   codex-rs/core/src/turn_diff_tracker.rs:

   struct TrackedContent { content: String, revision: u64 }
   baseline_by_path: HashMap<TrackedPath, TrackedContent>,
   current_by_path: HashMap<TrackedPath, TrackedContent>,
   

   Every tracked path keeps two full copies of the file (baseline + current) in
   RAM. The tracker has task lifetime (spans all turns — see
   codex-rs/core/src/session/turn.rs:258) and there is no maximum file size
   check: one 2.3 GB file adds ≥ 4.6 GB here alone; dozens of such files accumulate
   without eviction.

2. Entire accumulated diff rebuilt into one String on every patch —
   track_delta() → refresh_unified_diff() (turn_diff_tracker.rs:92..181)
   concatenates every cached per-file rendered diff into a single
   aggregated: String (aggregated.push_str(diff)) and stores it in
   self.unified_diff, on each delta. With ~13 GB of accumulated diff, every
   subsequent patch performs a fresh multi-GB allocation + memcpy (the old string
   frees after — hence the RSS sawtooth), i.e. repeated ~13–27 GB transient churn
   per edit.

3. The whole string is cloned and broadcast per event —
   codex-rs/core/src/tools/events.rs:605-657 (emit_patch_end) calls
   get_unified_diff(), which is self.unified_diff.clone()
   (turn_diff_tracker.rs:114), and sends it as
   EventMsg::TurnDiff(TurnDiffEvent { unified_diff }); the app-server forwards
   the full string again (app-server/src/bespoke_event_handling.rs:1207+).

4. The whole diff is also formatted into the log pipeline —
   codex-rs/tui/src/chatwidget/protocol_requests.rs:163:

   pub(super) fn on_turn_diff(&mut self, unified_diff: String) {
       debug!("TurnDiffEvent: {unified_diff}");   // logs the ENTIRE diff, every time
       self.refresh_status_line();
   }
   

   Each event formats/copies the full multi-GB string through tracing and (in this
   deployment) into the debug-log SQLite database.

5. The only "guard" makes things worse for big files —
   render_diff() uses similar::TextDiff::configure().timeout(100ms). For files
   far beyond what can be diffed in 100 ms, similar falls back to a coarse
   whole-file -all/+all hunk — the maximum possible diff size (matches the
   observed @@ -1,N +1,M @@ whole-file hunks and the near-zero context-line
   count). Each render also SHA-1 hashes the full baseline and current contents
   (git_blob_oid), which explains the continuous CPU burn.

Cost model per task editing a set of large files: Σ 2×(file size) permanently
pinned in the two content maps + Σ ≈2×(file size) in the cached rendered diffs,
plus a monotonically growing aggregated string that is re-materialized, cloned,
and logged per patch event → transient spikes of 2–4× the already multi-GB
live set on every edit, until the OOM killer wins.

Reproduction sketch

# 1. Make a large text file the tracker will follow:
seq 1 4000000 | sed 's/^/line /' > big.txt        # ~300 MB is plenty

# 2. In a codex session, repeatedly apply small patches to it
#    ("*** Update File: big.txt" …change a few lines…).
#
# 3. Watch codex RSS: the first touch makes the 100 ms diff timeout fall back to a
#    whole-file replace hunk (~2× file size of diff text retained); each further
#    patch re-renders and re-aggregates the accumulated diff, and every event
#    clones + logs the whole string. RSS multiplies quickly; with GB-scale inputs
#    the process is OOM-killed within hours.

Suggested fixes (any of 1–3 removes the unbounded growth)

1. Cap tracked content by size. For paths above a threshold (a few MB), record
   the change as metadata/placeholder ("large file changed, diff elided") instead
   of storing baseline/current contents and rendered diffs — or spill contents to
   temp files. The tracker's existing invalidate() fallback is a natural fit.
2. Don't rebuild/join/clone the accumulated diff per event. Produce the joined
   string lazily only when a subscriber asks, or emit per-file chunks
   incrementally; remove the per-patch get_unified_diff().clone() of an
   unbounded string.
3. Log sizes, not content, in on_turn_diff
   (debug!("TurnDiffEvent: {} bytes", unified_diff.len())).
4. When the diff computation times out (whole-file replace fallback), truncate the
   hunk body — -all/+all hunks carry no review value anyway.
5. A task-level byte budget for the tracker (content maps + rendered cache +
   aggregated string) with automatic invalidate() past the budget.

Impact

Any long agent session that edits large generated text files (build artifacts,
IR/log dumps, big fixtures, etc.) will exhaust memory and be OOM-killed.

What steps can reproduce the bug?

Reproduction sketch

# 1. Make a large text file the tracker will follow:
seq 1 4000000 | sed 's/^/line /' > big.txt        # ~300 MB is plenty

# 2. In a codex session, repeatedly apply small patches to it
#    ("*** Update File: big.txt" …change a few lines…).
#
# 3. Watch codex RSS: the first touch makes the 100 ms diff timeout fall back to a
#    whole-file replace hunk (~2× file size of diff text retained); each further
#    patch re-renders and re-aggregates the accumulated diff, and every event
#    clones + logs the whole string. RSS multiplies quickly; with GB-scale inputs
#    the process is OOM-killed within hours.

What is the expected behavior?

Not leaking memory and getting OOM killed

Additional information

No response

Start with codex-rs/core/src/turn_diff_tracker.rs and the task lifetime in codex-rs/core/src/session/turn.rs:258, then trace event handling through codex-rs/core/src/tools/events.rs:605-657 and codex-rs/tui/src/chatwidget/protocol_requests.rs:163. Run the documented large-file reproduction while watching RSS and debug-log growth. Done means repeated patches no longer retain or rebuild unbounded diff data or lead to OOM termination.