openai / openai/codex-plugin-cc

Harness auto-backgrounds foreground companion at Bash 120s timeout → reaped mid-turn, job wedged at 'running' (Linux repro; prompt-level fix insufficient — needs signal-flush, exitPromise race, PID liveness)

Open
#634 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
33.3k
Forks
2.3k
PR merge metrics
No merged PRs in 30d

Description

Summary

New forensic confirmation (Linux, 2026-08-10) of the "job wedged at running forever" family (#432, #391, #478) — with one new finding that makes the docs-only mitigation in #432 insufficient:

Even a fully compliant foreground invocation lands in the reaped-tree trap, because Claude Code auto-moves any foreground Bash call to a background shell after its 120s timeout. The subagent never set run_in_background; the harness did it for them. Once the subagent's turn ends, the orphaned background shell tree is reaped, the companion dies mid-turn by signal, runTrackedJob's catch never runs, and the job file is frozen at status: "running".

So the trigger cannot be closed by prompt instructions alone ("don't set run_in_background") — the runtime fixes are required.

Timeline (UTC, 2026-08-10, Linux)

Time Event
08:36:58 Rescue subagent runs codex-companion.mjs task --write --resume-last "…" as a foreground Bash call (retry after two server_overloaded failures that were correctly reported as failed)
08:37:01 Job task-msmz9xv7-iadon4 starts; thread resumed, turn 019fead1-… started
08:38:04 Last write ever to the job log and the harness's task output file ("File changes completed")
08:38:58–08:39:01 Bash call hits the 120s timeout → harness moves it to a background shell (ID: bwgcgq63k) and returns "You will be notified when it completes"
08:39:04 Subagent returns its stub ("running in background now… I'll return once notified") and exits
08:39:05.455 Codex rollout records the next apply_patch item — never logged by the companion → companion already dead in the 08:38:04–08:39:05 window
08:39–08:41 Server-side turn keeps going, applies 3 more patches, then ends with task_complete + error: server_overloaded at 08:41:01 — nobody is listening
08:39–09:33 Job file stays status: "running", phase: "editing"; the orchestrating session polls status for ~55 min, always seeing a live-looking corpse
09:33:43 Manual cancelturn/interrupt returns "no active turn to interrupt" (turn had ended 52 min earlier)

Negative checks: no OOM (dmesg clean), broker + codex app-server still alive and healthy afterwards (the next day's jobs ran fine on them), no crash output anywhere (worker stderr flows into the reaped shell's output file, which also stops at 08:38:04).

Impact

Same as #432/#391: wedged job blocks resolveLatestTrackedTaskThread ("Task … is still running"), status/status --wait report a corpse as live work, and the orchestrating session waits/polls indefinitely. Bonus hazard: the dying turn had already applied 4 patches to the working tree, so the repo is left half-modified with no failure signal.

Asks (all previously proposed; this issue adds a Linux repro showing prompt-level mitigation is not enough)

  1. Signal-flush in the companion/worker (per #432 proposal 2): SIGTERM/SIGHUP handler + process.on('exit') that writes status: "failed", errorMessage: "terminated by signal" to the job file before dying.
  2. Race state.completion against client.exitPromise in captureTurn (per #391/#390): today scripts/lib/codex.mjs awaits a promise resolved only by a turn/completed notification; a dropped broker socket resolves exitPromise but nothing rejects the completion — a second, independent silent-hang path.
  3. PID-liveness in status / buildSingleJobSnapshot (per #478 bug 2, #432 proposal 3): process.kill(pid, 0) → transition ESRCH'd running jobs to failed so a dead job can never present as live.
  4. (Docs, still worth doing) agents/codex-rescue.md: for anything that can exceed ~100s, pass the companion's own --background flag — a foreground call that outlives 120s will be auto-backgrounded into the reaped-tree trap no matter what the Bash tool parameters say.

Environment

  • Plugin: codex 1.0.1 (marketplace openai-codex)
  • Codex CLI: 0.145.0 · Claude Code (desktop app, remote/local agent) · Linux 6.8 (x86_64)
  • Trigger error on the server side: Selected model is at capacity / codex_error_info: "server_overloaded" ×3 consecutive turns (immediate retries; a backoff or model switch on this error class would also have avoided the third, fatal retry)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with scripts/lib/codex.mjs, especially captureTurn, and trace how completion and client.exitPromise interact when the broker socket closes. Then inspect the companion/worker termination handling and status/buildSingleJobSnapshot for PID liveness; reproduce the Linux timeout scenario if possible. Done means terminated, disconnected, and dead-PID jobs report failed rather than remaining running, with agents/codex-rescue.md documenting the long-running invocation guidance.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
devtools
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.