codex exec: concurrent first-run sessions against a fresh CODEX_HOME intermittently lose rows in state_5.threads
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 125k
- Forks
- 19.4k
- PR merge metrics
- PR metrics pending
Description
What version of Codex CLI is running?
codex-cli 0.150.1
What platform is your computer?
macOS (Darwin 25.5.0), arm64, APFS
What issue are you seeing?
When several codex exec processes start against a fresh CODEX_HOME at the same time, some of them never get a row in state_5.sqlite's threads table. Every process exits 0 and prints a correct answer, so nothing signals that a thread was dropped.
It does not reproduce once the CODEX_HOME has been used at least once, which is what makes me think it is the first-run initialization path rather than steady-state thread writing.
Measurement
Each run uses a brand-new CODEX_HOME with only auth.json copied in. Six sessions, each with a distinct prompt (Reply with exactly the number <i> and nothing else.). All six returned their own number and exited 0 in every condition below.
| condition | expected threads | state_5.threads |
sessions/*.jsonl |
|---|---|---|---|
| 6 sessions, one at a time | 6 | 6 | 6 |
| 6 concurrent, warmed home (one session run first) | 7 | 7 | 7 |
| 6 concurrent, fresh home — run 1 | 6 | 1 | 6 |
| 6 concurrent, fresh home — run 2 | 6 | 6 | 6 |
| 6 concurrent, fresh home — run 3 | 6 | 5 | 6 |
Three things stand out:
- The rollout files are always complete.
sessions/*.jsonlhas every session in every run — one append-only file per session, so nothing there can collide. The loss is confined to the SQLite thread index. - It is intermittent, not deterministic: 5 lost, then 0, then 1.
- Warming the home is enough to avoid it. A single prior session makes the same 6-way concurrency clean.
Reproduction
#!/bin/zsh
H=$(mktemp -d)/codex-home
mkdir -p "$H" && cp ~/.codex/auth.json "$H/"
W=$(mktemp -d)
cd "$W"
for i in $(seq 1 6); do
( CODEX_HOME=$H codex exec --skip-git-repo-check \
"Reply with exactly the number $i and nothing else." >/dev/null 2>&1
echo "session $i exit=$?" ) &
done
wait
python3 - "$H" <<'PY'
import sqlite3, sys, os, glob
h = sys.argv[1]
con = sqlite3.connect(os.path.join(h, "state_5.sqlite"), timeout=30)
print("threads in state_5.sqlite :", con.execute("SELECT COUNT(*) FROM threads").fetchone()[0])
print("rollout jsonl files :", len(glob.glob(os.path.join(h, "sessions", "**", "*.jsonl"), recursive=True)))
PY
Expected: threads = 6. Observed: 1, 6, and 5 on three consecutive fresh homes.
Prepending a single warm-up run before the loop makes it consistently correct:
CODEX_HOME=$H codex exec --skip-git-repo-check "Reply with exactly: warmup" >/dev/null 2>&1
Why it matters
"Several agents start at once against a CODEX_HOME that does not exist yet" is not an unusual state — it is a fresh container, a CI job, or a new machine running more than one Codex in parallel, which is exactly when people reach for parallel agents. The user-visible effect is that some sessions are missing from history and cannot be resumed, with no error at the time and no way afterwards to tell a dropped thread from one that was never started.
The rollout JSONL surviving intact suggests the data is recoverable in principle, so this may be reconcilable rather than a true data loss.
What I did not measure
memories_1.sqlite (stage1_outputs) stayed at 0 rows in every condition — these sessions were too short to trigger memory distillation, so I have no data on the memory pipeline under concurrency. This report is only about the thread index.
Possibly related
#38150 (thread writer lock directory under CODEX_HOME) and #31184 (database is locked) touch the same storage layer, though both describe a shared or long-lived home rather than first-run initialization.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the six-way concurrent run against a fresh CODEX_HOME and compare state_5.sqlite's threads count with the sessions/*.jsonl files. Trace the first-run initialization path and thread-index writes, using issues #38150 and #31184 for related storage behavior. Done means every concurrent session has a threads row without regressing warmed-home behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sqlite
- Domain
- cli, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 56/100