openai / openai/codex

[CLI] Active rollout writer keeps writing to an unlinked inode after the session path is replaced

Open
#38,149 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug CLI session
Dominant language
Rust
Stars
125k
Forks
19.5k
PR merge metrics
PR metrics pending

Description

What version of Codex CLI is running?

Observed with codex-cli 0.143 and onwards

What subscription do you have?

N/a

Which model were you using?

Model-independent; this occurs in the local rollout persistence layer.

What platform is your computer?

Linux 6.16

What terminal emulator and version are you using?

Not terminal-specific.

Codex doctor report

N/A

What issue are you seeing?

Codex keeps the file descriptor for an active session rollout JSONL open across turns. If a synchronization, backup, or restore tool replaces the rollout pathname using normal Unix unlink-and-copy or rename-over semantics, the pathname starts referring to a new inode while Codex keeps writing to the old, unlinked inode.

Those later writes and flushes succeed, so Codex emits no error. The visible rollout path remains stale, and when the Codex process exits the unlinked inode—and every turn appended to it after replacement—disappears. codex resume <thread-id> then reloads the stale file and appears to go backward in time.

This is not specific to one synchronization product; any process that replaces $CODEX_HOME/sessions/...jsonl while a thread is live can trigger it.

What steps can reproduce the bug?

Use a disposable thread because this intentionally demonstrates loss of its post-replacement tail.

  1. On Linux, start a new interactive Codex thread and send FIRST_SENTINEL. Record the thread UUID from /status or its rollout filename.
  2. In a second terminal, replace that rollout path with a byte-identical file on a different inode:
THREAD_ID='<disposable-thread-uuid>'
CODEX_HOME_DIR="${CODEX_HOME:-$HOME/.codex}"
ROLLOUT="$(find "$CODEX_HOME_DIR/sessions" -type f -name "*${THREAD_ID}.jsonl" -print -quit)"
test -n "$ROLLOUT"
OLD_ID="$(stat -c '%d:%i' "$ROLLOUT")"
cp --preserve=all "$ROLLOUT" "$ROLLOUT.replacement"
mv -f "$ROLLOUT.replacement" "$ROLLOUT"
NEW_ID="$(stat -c '%d:%i' "$ROLLOUT")"
printf '%s -> %s\n' "$OLD_ID" "$NEW_ID"
test "$OLD_ID" != "$NEW_ID"
  1. In the still-running Codex thread, send SECOND_SENTINEL_AFTER_REPLACEMENT and wait for the response.
  2. Inspect the visible rollout path:
grep -F 'SECOND_SENTINEL_AFTER_REPLACEMENT' "$ROLLOUT"

The command finds no match even though the turn completed successfully. While Codex is still running, lsof +L1 can also show its writable descriptor for the deleted rollout inode.

  1. Exit Codex and run codex resume "$THREAD_ID". The resumed transcript contains the first turn but not the second.

What is the expected behavior?

Codex should not report a successful durable write to a rollout file that is no longer reachable through the configured pathname. After the second prompt, the canonical rollout path and a resumed thread should contain that turn.

If Codex cannot safely reconcile a replacement with the live branch, it should surface the conflict and preserve the local tail at a recoverable path rather than silently losing it. Reopening the current pathname—or comparing the open descriptor's (st_dev, st_ino) with the pathname and then safely reopening/forking—before each non-empty durable batch would detect this case.

Additional information

The current writer lifecycle in codex-rs/rollout/src/recorder.rs appears to explain the behavior:

  • RolloutWriterState caches Option<JsonlWriter>.
  • ensure_writer_open returns immediately when that option is already populated.
  • write_pending_once writes and flushes but does not close or reopen the file.
  • enter_recovery_mode drops the writer only after an I/O error. Writing an unlinked Unix inode remains valid, so pathname replacement does not trigger it.
  • The background rollout_writer task owns this state until shutdown.

The source comments explain the background task as a way to keep writes asynchronous and ordered, but I could not find a documented requirement to retain the same descriptor across successful write batches. Is that descriptor lifetime intentional? If so, what invariant requires it?

There is already a reopen-per-call precedent for unloaded threads: append_rollout_item_to_path opens the current path, derives its ordinal state, appends, and then drops the local writer.

A focused regression test could persist item A, replace the active rollout with byte-identical contents on a new inode, persist and flush item B, and assert that the currently named rollout contains B with valid ordinals.

For comparison, a controlled single-session probe of Claude Code 2.1.220 on Linux found no transcript JSONL descriptor held open between turns. After a byte-identical path replacement, its next turn appended to the replacement inode. That does not make Claude immune to a short write-time race, but it avoids this persistent active-writer split.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in codex-rs/rollout/src/recorder.rs, especially RolloutWriterState, ensure_writer_open, write_pending_once, and enter_recovery_mode; compare them with append_rollout_item_to_path. Add a focused regression test that replaces an active rollout with byte-identical contents on a new inode before persisting item B. Done means the named rollout and a resumed thread retain item B with valid ordinals, or the replacement conflict is surfaced without silent loss.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.