openai / openai/codex-plugin-cc
test: broker-spawn integration tests flake under host load — widen/tune the waitFor budget and reap leaked processes
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 33.3k
- Forks
- 2.3k
- PR merge metrics
- No merged PRs in 30d
Description
Update (2026-08-10): the primary failure mode originally described here turned out to be a product race, not test-infrastructure flakiness — #623's enqueue path rewrote the job file (truncate-in-place) at the exact moment the detached worker's startup read it, killing the worker silently and leaving the job
queuedforever; host load only modulated the collision probability. That race is fixed in #623 (writeJobFilenow writes a temp file and renames it into place, with a torn-record regression test), and the suite passes 144/144 on the host that showed the failures. This issue's remaining scope is the two test-infrastructure hardenings below, which stand on their own.
Problem
The runtime tests that spawn the full background-task chain — detached worker → workspace broker → fake codex app-server → unix-socket handshake → turn/start — gate on waitFor(..., { timeoutMs: 15000 }). On a loaded host (a VM reserving most cores, an endpoint-security agent scanning every spawned process), that chain can exceed 15 s even with no functional defect, and the test times out. Observed on 2026-08-10 while validating #623: even after the product race above was fixed, one run had a test miss the budget by ~1.2 s purely from suite parallelism plus per-spawn scanning overhead.
A compounding factor: a test run that fails or is interrupted leaks its temp-workspace broker and app-server processes (codex-plugin-test-* cwd). These accumulated to ~150 idle node processes across a few failed runs, degrading subsequent runs until manually swept with pkill -f codex-plugin-test-. A near-green run (142/143) still left 66 such processes behind, so teardown is best-effort even on the passing path.
Proposal
- Make the wait budget scalable: apply an env multiplier (e.g.
CODEX_COMPANION_TEST_TIMEOUT_SCALE) inside thewaitForhelper (tests/runtime.test.mjs:20) so loaded hosts and slow CI runners can widen every budget in one place, keeping the defaults for fast machines. - Reap spawned processes on test teardown unconditionally (including timeout and passing paths) so runs cannot leak workspace processes and poison later runs.
Out of scope
Product-code changes. The torn-read race originally blamed on load is fixed in #623; the spawn chain's inherent cost is by design for these integration-level tests.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in tests/runtime.test.mjs at the waitFor helper around line 20, then trace how the detached worker, broker, and app-server processes are spawned and torn down. Add the configurable timeout scaling and ensure teardown runs on failures, interruption, and passing paths; done means the runtime suite tolerates slower hosts without leaked codex-plugin-test-* processes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100