BOHICA-LABS / BOHICA-LABS/vsdd-factory

per-story-delivery: no hardware-execution smoke gate for stories with hardware-gated ACs

Open
#543 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2
Forks
1
Avg merge
6h 43m
Merged PRs (30d)
29

Description

## Observed

A story wiring a CLI to real FIDO2 hardware (libfido2 make-credential FFI) completed the full per-story pipeline — TDD with mock-backend CI coverage, 8 fresh-context adversarial passes reaching the 3-consecutive-CLEAN convergence gate, per-AC demo evidence (mock-seam), full PR lifecycle with green CI, squash-merge — without the hardware path ever executing once. The story's hardware ACs had properly \`#[ignore]\`-gated integration tests (correct CI hygiene), but nothing in the pipeline required running them even once on real hardware before convergence or merge.

The first real-device invocation after merge failed immediately: \`FIDO_ERR_PIN_REQUIRED\`. A PIN-protected authenticator is the common configuration for modern FIDO2 keys, so this was a happy-path failure, not an edge case. The human operator's scheduled hardware-evaluation window was burned on the discovery; a fix story is now required.

## Engine gap

The pipeline's mock-seam discipline is correct for CI, but mock-green + adversarial-review-of-code cannot validate assumptions about real device behavior. Adversarial passes explicitly annotated "mock-vs-real divergence boundaries" as out of their reach ("only hardware-verifiable via the #[ignore] test") — yet no step forces that boundary to be crossed before ship. Convergence and merge are declared on a surface whose defining half has never run.

Notably, this is the SECOND consecutive story on the same product where the hardware-blocked gap was discovered at the operator's keyboard post-merge (the first is #538's AC-stub case). The pipeline repeatedly ships hardware claims that fail at first touch.

## Suggested fix

Per-story-delivery: when a story's AC table contains hardware-gated (\`#[ignore]\`/manual) test gates, add a mandatory pre-merge (or pre-convergence-declaration) step: "hardware smoke run" — the operator executes at minimum the hardware happy path once against a real device and the result is recorded in the story's delivery evidence. Cheap (single command, <5 min), catches whole classes of library/device-reality mismatches while the fix cycle is still open. If no hardware is available, the gate is explicitly waived by the human with the deferral recorded — not silently skipped.

## Evidence

Sessions 2026-07-06/07, private consumer project: story converged 3/3 CLEAN over 8 passes, merged with 7 green CI checks; first real-device run failed with FIDO_ERR_PIN_REQUIRED (PIN-protected authenticator — default posture for the device class).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.