BOHICA-LABS / BOHICA-LABS/vsdd-factory
process-gap(tdd): security-invariant ACs require mutation verification — a test that still passes when the guard is removed does not test the guard
- Lingua principale
- Rust
- Stelle
- 2
- Fork
- 1
- Merge medio
- 6h 43m
- PR unite (30g)
- 29
Descrizione
## Problem
A security-critical guard was implemented as two separable pieces — a pure decision function and a state-read helper — but was never composed at the production call site. The code was dead in production.
Its regression tests passed against the unwired code. The reasons were structural:
- One assertion used a count that was satisfiable by zero entries (e.g. `<= 1` is satisfied by 0).
- A liveness check ("did not hang / request left the queue") holds regardless of whether the guard fired.
Three independent fresh-context adversarial passes (different model, no prior-pass context) all flagged the same issue as a `[process-gap]`. The guard was implemented; the wiring was not.
The story itself documented deferring the decisive "effect-not-observed" proof to a downstream story — a deferred wiring obligation that the pipeline accepted as a valid close condition.
## What fixed it
1. A deterministic test that forces the exact guarded window via a cfg-gated ordering seam, making the guard's effect observable in isolation.
2. **Mutation verification** as the acceptance bar: neutralize the guard (e.g. comment out the call or replace the decision with `true`), confirm the test FAILS; restore, confirm it passes.
Only after both checks was the guard considered verified.
## Proposed engine improvement
VSDD test-integrity guidance for **security-invariant ACs** should:
1. **Require mutation verification**: the test suite for a security-invariant AC must include at least one test that is explicitly mutation-verified — i.e., confirmed to FAIL when the guarded behavior is removed (the guard neutralized), and confirmed to PASS when it is restored.
2. **Prohibit deferring the wiring proof**: a story that implements a security guard must verify, within the same story, that the guard is actually called on the production path. Deferring the "effect-not-observed" proof to a downstream story is not a valid close condition for a security-invariant story.
3. **Adversary guidance**: flag any security-invariant AC whose tests are satisfiable without the guard (i.e., the assertion holds on empty/zero/no-effect output). This is the structural signature of an unverified guard.
## Why this matters
The gap is not visible in diff review: the guard code exists, it compiles, the tests pass. The adversary reviewing the diff sees a guard implementation. What neither the adversary nor the test suite sees is that the production call site was never changed to call it.
Mutation verification closes this gap because it makes the guard's presence causally necessary for the test to pass — not merely incidentally present in the diff.
## Relationship to existing guidance
Mutation testing is already present in the Phase 6 formal hardening phase (cargo-mutants / similar). This issue proposes moving a targeted mutation check **earlier** — into the TDD story's own AC verification requirements — specifically for security-invariant ACs, where the cost of deferring is highest.
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
The issue describes a process gap in test-driven development for security invariants. It proposes changes to the VSDD engine's test-integrity guidance. To start, examine the existing test framework and AC (Acceptance Criteria) verification logic in the codebase. Look for where security-invariant tests are defined and how mutation verification could be integrated. The goal is to design a requirement that tests must fail when the security guard is neutralized.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- rust
- Ambito
- security, testing-qa
- Tipo di issue
- Funzionalità
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Stato di attività
- Tranquilla
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 30/100