BOHICA-LABS / BOHICA-LABS/vsdd-factory
process-gap(adversarial-refinement): race-finding fixes that install sync barriers must fail closed and be stress-validated under constrained schedulers
- Dominant language
- Rust
- Stars
- 2
- Forks
- 1
- Avg merge
- 6h 43m
- Merged PRs (30d)
- 29
Description
## Context
Project: switchboard-blue (Go, greenfield cycle-1, currently post-SHIPPED steady state, plugin 1.0.0-rc.21).
A metadata-only PR (`.run.yaml`, zero code changes) failed the Quality Gate **twice in a row** on two *different* pre-existing flaky tests in the same package — both scheduling races that shared `-race` CI runners widen. Both tests were produced/adjusted by factory-dispatched agents during Phase 3/5.
## The pattern
**Instance 1 — fail-open sync barrier.** A Pass-3 L2 adversarial race finding correctly flagged a `time.Sleep`+`Gosched` synchronization in a test (register-before-serve invariant, verifying an atomic flag set at Serve entry). The dispatched fix replaced it with a dial-until-accept loop — the right idea — but:
1. The dial loop fast-fails on connection-refused (microseconds per attempt), so all 50 attempts can complete before the Serve goroutine is ever scheduled. No sleep between attempts.
2. When the barrier failed to establish, the test **logged and asserted anyway** (`t.Log("...may be flaky")` then proceed), producing a false invariant-violation failure instead of a loud barrier failure.
The fix for a race finding introduced a new race with a fail-open observability hole.
**Instance 2 — success-path race in a shutdown-drop test.** A Ruling-compliance test (connection accepted post-Shutdown must be dropped) reads from the client half of a `net.Pipe` with a deadline. If the server drops the connection **fast enough** — i.e., the exact behavior the test verifies — `SetReadDeadline` returns `io.ErrClosedPipe` and the test `t.Fatal`'d on its own PASS condition.
## Why it slipped through (introspection)
- Both tests are green on fast dev machines; the windows only open under `-race` + shared-runner scheduler starvation.
- The adversary reviews test *logic* vs spec, but no pass exercises tests under scheduler pressure. A race-finding *fix* is accepted if it removes the flagged primitive (`time.Sleep`), not if the replacement barrier is proven sound.
- No convergence-dimension check catches "test fails on its own success condition" (instance 2) — that requires reasoning about the timing of the behavior under test relative to the observation channel.
## Proposed improvements
1. **Fail-closed rule for sync barriers in dispatched race fixes:** when test-writer/implementer installs a synchronization barrier in a test (dial loop, channel wait, condition poll), the barrier must (a) yield between attempts, and (b) `t.Fatal` with a barrier-specific message if it never establishes — never log-and-assert-anyway. Candidate for the test-writer agent prompt and the adversarial race-finding fix-routing instructions.
2. **Stress validation gate for race-finding fixes:** the fix-PR evidence for any race/flake finding should include a constrained-scheduler stress run (e.g. `GOMAXPROCS=2 go test -race -count=10 -run `), not just a single green run.
3. **Adversarial axis candidate:** "observation-channel timing" — for tests that observe an asynchronous behavior through a channel/pipe/socket, ask whether the observed behavior *completing quickly* can break the observation mechanism itself (instance 2's class).
## Evidence
- Quality Gate run 28732071943: attempt 1 failed `TestRegister_AfterServeReturnsError`, attempt 2 failed `TestServe_DrainCompletesWithinBudget_RulingI`, attempt 3 green.
- Fix PR (test-only, both instances): ArcavenAE/switchboard-blue#84.
Severity: MEDIUM (process-gap — factory-authored race fixes can institutionalize new flakes; erodes trust in the Quality Gate signal and burns rerun cycles on every downstream PR).
Contributor guide
Assessment
This issue has not been assessed yet.