bin/gstack-codex-probe: bash-native watchdog returns 143 instead of 124 under load (flaky codex-hardening test)
- Dominant language
- TypeScript
- Stars
- 133k
- Forks
- 19.9k
- Avg merge
- 18h 46m
- Merged PRs (30d)
- 26
Description
## Summary
`_gstack_codex_timeout_wrapper` in `bin/gstack-codex-probe` has a timing race in its bash-native fallback (the path taken on stock macOS with no `gtimeout`/`timeout`). When the watchdog fires, the wrapper sometimes returns the raw SIGTERM code 143 instead of the documented 124. This makes `test/codex-hardening.test.ts › bash-native watchdog kills a hung command at the deadline (exit 124, no timeout binary)` flaky under `bun run test:free` with 6 parallel shards.
Seen on v1.81.0.0 (05303928); the wrapper is identical in v1.80.0.0, so this is pre-existing, not a regression.
## Observed
```
error: expect(received).toContain(expected)
Expected to contain: "rc=124"
Received: "rc=143\n"
at test/codex-hardening.test.ts:307:24
(fail) gstack-codex-probe: timeout wrapper + namespace hygiene > bash-native watchdog kills a hung command at the deadline (exit 124, no timeout binary) [1014.16ms]
```
macOS 15 / Darwin 25.6.0, arm64, /bin/bash 3.2.57, bun 1.3.14. Passes reliably when run alone; failed once out of three full 6-shard `test:free` runs on an M-series MacBook Air.
## Root cause
```bash
"$@" &
local _cmd_pid=$!
( sleep "$_duration" && kill -TERM "$_cmd_pid" 2>/dev/null ) >/dev/null 2>&1 &
local _watch_pid=$!
wait "$_cmd_pid"
_rc=$?
if kill -0 "$_watch_pid" 2>/dev/null; then
# treated as "command finished before the deadline"
kill "$_watch_pid"; wait "$_watch_pid"
elif [ "$_rc" -ge 128 ]; then
_rc=124
fi
```
After the watchdog subshell sends `kill -TERM`, two things happen concurrently: the parent's `wait` returns, and the subshell exits. If the parent runs `kill -0 "$_watch_pid"` before the subshell has exited (or before bash has reaped it, since `kill -0` succeeds on a zombie), the wrapper takes the "finished early" branch and returns 143. Under CPU contention the subshell is often descheduled right after its `kill` syscall, which is exactly when the parent wakes up.
Deterministic reproduction, holding the watchdog alive for 20 ms after its kill:
```bash
# same code as the wrapper, with `; sleep 0.02` appended inside the watchdog subshell
delay 0.02: watchdog alive -> early-finish branch rc=143
```
## Suggested fix
Don't infer "killed by the watchdog" from whether the watchdog subshell is still alive. Either:
1. Record the start time and map to 124 when the command died by signal **and** the deadline has elapsed:
```bash
local _start=$SECONDS
...
wait "$_cmd_pid"; _rc=$?
kill "$_watch_pid" 2>/dev/null; wait "$_watch_pid" 2>/dev/null
if [ "$_rc" -ge 128 ] && [ $((SECONDS - _start)) -ge "$_duration" ]; then _rc=124; fi
```
2. Or have the watchdog leave an explicit marker (a temp file or a fd write) after it kills, and check that.
Either removes the dependence on process-exit ordering. Happy to open a PR if you'd like.
Contributor guide
Research direction
Start in bin/gstack-codex-probe at _gstack_codex_timeout_wrapper, then read test/codex-hardening.test.ts around line 307 and run the focused bash-native watchdog test. Reproduce under parallel bun run test:free conditions or with the described delayed watchdog, then verify the wrapper consistently reports exit 124 and the existing test passes without a timeout binary.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bash, bun, typescript
- Domain
- cli, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100