BOHICA-LABS / BOHICA-LABS/vsdd-factory
per-story-delivery: no hardware-execution smoke gate for stories with hardware-gated ACs
- Dominant language
- Rust
- Stars
- 2
- Forks
- 1
- Avg merge
- 6h 43m
- Merged PRs (30d)
- 29
Description
## Observed
A story wiring a CLI to real FIDO2 hardware (libfido2 make-credential FFI) completed the full per-story pipeline — TDD with mock-backend CI coverage, 8 fresh-context adversarial passes reaching the 3-consecutive-CLEAN convergence gate, per-AC demo evidence (mock-seam), full PR lifecycle with green CI, squash-merge — without the hardware path ever executing once. The story's hardware ACs had properly \`#[ignore]\`-gated integration tests (correct CI hygiene), but nothing in the pipeline required running them even once on real hardware before convergence or merge.
The first real-device invocation after merge failed immediately: \`FIDO_ERR_PIN_REQUIRED\`. A PIN-protected authenticator is the common configuration for modern FIDO2 keys, so this was a happy-path failure, not an edge case. The human operator's scheduled hardware-evaluation window was burned on the discovery; a fix story is now required.
## Engine gap
The pipeline's mock-seam discipline is correct for CI, but mock-green + adversarial-review-of-code cannot validate assumptions about real device behavior. Adversarial passes explicitly annotated "mock-vs-real divergence boundaries" as out of their reach ("only hardware-verifiable via the #[ignore] test") — yet no step forces that boundary to be crossed before ship. Convergence and merge are declared on a surface whose defining half has never run.
Notably, this is the SECOND consecutive story on the same product where the hardware-blocked gap was discovered at the operator's keyboard post-merge (the first is #538's AC-stub case). The pipeline repeatedly ships hardware claims that fail at first touch.
## Suggested fix
Per-story-delivery: when a story's AC table contains hardware-gated (\`#[ignore]\`/manual) test gates, add a mandatory pre-merge (or pre-convergence-declaration) step: "hardware smoke run" — the operator executes at minimum the hardware happy path once against a real device and the result is recorded in the story's delivery evidence. Cheap (single command, <5 min), catches whole classes of library/device-reality mismatches while the fix cycle is still open. If no hardware is available, the gate is explicitly waived by the human with the deferral recorded — not silently skipped.
## Evidence
Sessions 2026-07-06/07, private consumer project: story converged 3/3 CLEAN over 8 passes, merged with 7 green CI checks; first real-device run failed with FIDO_ERR_PIN_REQUIRED (PIN-protected authenticator — default posture for the device class).
Contributor guide
Assessment
This issue has not been assessed yet.