openai / openai/codex

codex exec: concurrent first-run sessions against a fresh CODEX_HOME intermittently lose rows in state_5.threads

Open
#42,447 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug CLI exec session
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

What version of Codex CLI is running?

codex-cli 0.150.1

What platform is your computer?

macOS (Darwin 25.5.0), arm64, APFS

What issue are you seeing?

When several codex exec processes start against a fresh CODEX_HOME at the same time, some of them never get a row in state_5.sqlite's threads table. Every process exits 0 and prints a correct answer, so nothing signals that a thread was dropped.

It does not reproduce once the CODEX_HOME has been used at least once, which is what makes me think it is the first-run initialization path rather than steady-state thread writing.

Measurement

Each run uses a brand-new CODEX_HOME with only auth.json copied in. Six sessions, each with a distinct prompt (Reply with exactly the number <i> and nothing else.). All six returned their own number and exited 0 in every condition below.

condition expected threads state_5.threads sessions/*.jsonl
6 sessions, one at a time 6 6 6
6 concurrent, warmed home (one session run first) 7 7 7
6 concurrent, fresh home — run 1 6 1 6
6 concurrent, fresh home — run 2 6 6 6
6 concurrent, fresh home — run 3 6 5 6

Three things stand out:

  • The rollout files are always complete. sessions/*.jsonl has every session in every run — one append-only file per session, so nothing there can collide. The loss is confined to the SQLite thread index.
  • It is intermittent, not deterministic: 5 lost, then 0, then 1.
  • Warming the home is enough to avoid it. A single prior session makes the same 6-way concurrency clean.
Reproduction
#!/bin/zsh
H=$(mktemp -d)/codex-home
mkdir -p "$H" && cp ~/.codex/auth.json "$H/"
W=$(mktemp -d)

cd "$W"
for i in $(seq 1 6); do
  ( CODEX_HOME=$H codex exec --skip-git-repo-check \
      "Reply with exactly the number $i and nothing else." >/dev/null 2>&1
    echo "session $i exit=$?" ) &
done
wait

python3 - "$H" <<'PY'
import sqlite3, sys, os, glob
h = sys.argv[1]
con = sqlite3.connect(os.path.join(h, "state_5.sqlite"), timeout=30)
print("threads in state_5.sqlite :", con.execute("SELECT COUNT(*) FROM threads").fetchone()[0])
print("rollout jsonl files       :", len(glob.glob(os.path.join(h, "sessions", "**", "*.jsonl"), recursive=True)))
PY

Expected: threads = 6. Observed: 1, 6, and 5 on three consecutive fresh homes.

Prepending a single warm-up run before the loop makes it consistently correct:

CODEX_HOME=$H codex exec --skip-git-repo-check "Reply with exactly: warmup" >/dev/null 2>&1
Why it matters

"Several agents start at once against a CODEX_HOME that does not exist yet" is not an unusual state — it is a fresh container, a CI job, or a new machine running more than one Codex in parallel, which is exactly when people reach for parallel agents. The user-visible effect is that some sessions are missing from history and cannot be resumed, with no error at the time and no way afterwards to tell a dropped thread from one that was never started.

The rollout JSONL surviving intact suggests the data is recoverable in principle, so this may be reconcilable rather than a true data loss.

What I did not measure

memories_1.sqlite (stage1_outputs) stayed at 0 rows in every condition — these sessions were too short to trigger memory distillation, so I have no data on the memory pipeline under concurrency. This report is only about the thread index.

Possibly related

#38150 (thread writer lock directory under CODEX_HOME) and #31184 (database is locked) touch the same storage layer, though both describe a shared or long-lived home rather than first-run initialization.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the six-way concurrent run against a fresh CODEX_HOME and compare state_5.sqlite's threads count with the sessions/*.jsonl files. Trace the first-run initialization path and thread-index writes, using issues #38150 and #31184 for related storage behavior. Done means every concurrent session has a threads row without regressing warmed-home behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sqlite
Domain
cli, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
56/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.