BOHICA-LABS / BOHICA-LABS/vsdd-factory

process-gap(test-writer+adversary): tests spawning shells/interactive-capable subprocesses inherit stdin from the runner — hangs interactively, false-green on CI, .stdin(null) not mandated

Open
#424 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2
Forks
1
Avg merge
6h 43m
Merged PRs (30d)
29

Description

## Process-gap

When a test spawns a shell or any subprocess that *could* enter an interactive read on stdin, common test-driver crates (Rust \`assert_cmd\` / \`std::process::Command\`, Go \`exec.Command\`, Node \`spawn\`) default to inheriting the runner's stdin. In CI (no TTY, stdin closed) this is invisible; run the same test locally in a terminal and the subprocess blocks waiting for input, and \`cargo test\` / \`go test\` hangs until the harness times out or the human aborts.

The hazard is a Sev-flake: green on CI, hang locally, no diagnostic. Worse, the hang can be misdiagnosed as a product regression when the actual defect is the test's stdin plumbing.

## Concrete shape (sanitized)

A CLI-integration test spawned a subprocess whose fallback path executes \`/bin/sh\`. On CI (piped stdin) the shell exits immediately (EOF); locally in a TTY it prints a prompt and waits. The fix was a one-liner: \`.stdin(Stdio::null())\` on the \`Command\` builder before \`.output()\`. Adversary flagged the missing null-stdin as a MEDIUM finding during pass-1 review, but only after a local test-suite run had actually hung.

## Why gates miss it

- \`cargo test\` on CI has no TTY, so the failure mode is invisible.
- Adversary reviews test *logic* (assertion strength, coverage) — not test *plumbing* (stdin/env/timeouts).
- test-writer prompt does not enumerate "for every subprocess-spawning test, set stdin explicitly."
- No lint exists that flags \`Command::new(...).output()\` on a shell-capable target without an explicit stdin setting.

## Proposed remediation

1. **test-writer** dispatch checklist addition (paraphrased): "For any test that spawns a subprocess capable of interactive input (any shell, any REPL, any TUI, any command that may prompt), explicitly set stdin to the equivalent of /dev/null (\`.stdin(Stdio::null())\` in Rust; in Node pipe \`'ignore'\`; in Go set \`cmd.Stdin\` to an already-closed reader — \`cmd.Stdin = nil\` inherits, which is the trap). Also set a per-command timeout. Never inherit from the test runner."
2. **adversary** test-plumbing check: scan test files for \`Command\` / \`spawn\` / \`exec\` invocations that don't set stdin *and* target a shell-name or CLI-that-can-prompt.
3. **lint** (optional, Rust-specific): a \`cargo clippy\` custom-lint or ripgrep-driven CI check to flag missing \`.stdin(...)\` on \`assert_cmd::Command\` targeting shells.

## Distinct from adjacent issues

- **#359** (platform-specific tests silently broken on macOS-pinned CI) — different axis: that's OS-conditional test compilation; this is TTY/no-TTY runtime asymmetry.
- **#357** (Red Gate tests can't land — CI rejects idiomatic test lints) — different: CI rejection at merge, not runtime hang.
- **#330** (headless test runs pass green while blind to render/input) — closest cousin, but the modality there is *render*; here the modality is *stdin*. Same shape at abstract level (environment discrepancy between CI and human-driven test runs), but different practical remediation. Could be filed as a sibling under a shared "environment-discrepancy hazards" umbrella.

## Impact

One hang per developer-per-session is small; but agentic test-writers emit this pattern by default, and every new subprocess-spawning test carries the same latent hang. Compounded across a project's lifetime this is minutes-to-hours of lost time, plus a false-alarm rate when the hang is misread as a product regression.

## Severity

MEDIUM. Not architectural; but structurally recurs and hides behind CI's null TTY environment.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.