BOHICA-LABS / BOHICA-LABS/vsdd-factory
bug(story-writer+test-writer): an AC sub-case that contradicts its anchored BC, plus a test whose NAME asserts the opposite of its assertion, survive all spec + TDD gates
- Dominant language
- Rust
- Stars
- 2
- Forks
- 1
- Avg merge
- 6h 43m
- Merged PRs (30d)
- 29
Description
## Summary & impact
A story acceptance-criterion sub-case can prescribe an outcome that (a) contradicts another sub-case of the same AC and (b) contradicts the anchored behavioral contract — and survive spec review, consistency validation, and multiple adversarial rounds. The contradiction then gets silently encoded into the test suite as a test whose **name implies one outcome while its assertion checks the opposite**. The implementation happens to be correct (it follows the BC), so all gates go green — but the spec and the test name now actively mislead any future maintainer, and only a fresh-context adversarial lens caught it after 4+ convergence rounds.
## Conceptual breakdown
Two layers fail together:
1. **Story-writer / spec gate.** An AC enumerates sub-cases (a)…(d). Sub-case (a) prescribes an outcome that is logically impossible to distinguish from sub-case (c) at the validation layer (the two inputs are indistinguishable in the implementation language — e.g. an explicit empty string vs. an absent field both decode to the language's zero value). Sub-case (a) also contradicts the anchored BC's postcondition. No spec-time gate cross-checks (i) sub-cases of one AC against each other for mutual satisfiability, or (ii) each AC sub-case against the anchored BC clause it cites.
2. **Test-writer / TDD gate.** The test author correctly implements the BC behavior (so the test passes against correct code) but **names the test case after the wrong AC prose**. Result: a test case named to imply outcome X (e.g. `..._rejected`) whose assertion checks outcome NOT-X (e.g. `wantErr: false`), with a confused comment trying to reconcile the two. The "all tests pass" convergence gate trivially accepts this — the assertion is correct; only the name lies. No gate cross-checks a test-case's name-implied outcome against its actual assertion polarity.
## Why gates miss it
- Consistency-validator checks references/links, not logical satisfiability of AC sub-cases against each other.
- The adversary catches it only on a fresh-context lens that re-derives the BC↔AC mapping from scratch; earlier passes anchored on the (wrong) AC prose and did not re-check it against the BC.
- The green test suite is genuinely green — the defect is in the NAME and the SPEC PROSE, not the assertion. Coverage and pass-rate metrics are blind to it.
This is adjacent to the long-standing "tautological / false-coverage test" concern, but it is a distinct failure mode: here the assertion is *correct*; the **name and the spec prose are wrong** and point the opposite way.
## Proposed mitigations
- **Spec-time gate (story-writer / pre-Phase-3 self-containment gate):** for each AC, cross-check every sub-case against (i) the other sub-cases of the same AC for mutual satisfiability given the implementation language's type semantics, and (ii) the specific anchored BC clause it cites. Flag a sub-case whose prescribed outcome contradicts the BC or another sub-case.
- **Test-writer / convergence gate:** flag a test case whose name encodes an outcome polarity (`_rejected`, `_accepted`, `_fails`, `_ok`, `_error`, `_valid`) that disagrees with its assertion polarity (`wantErr`, expected-error presence). Cheap static heuristic; high signal.
- **Adversary axis:** add an explicit "test-name ↔ assertion polarity" check and an "AC sub-case mutual-satisfiability" check to the spec-fidelity lens so it isn't dependent on a fresh instance happening to re-derive the BC mapping.
## Repro signal
| Field | Value |
|-------|-------|
| Plugin | vsdd-factory 1.0.0-rc.21 |
| Claude Code | 2.1.190 |
| Model | claude-opus-4-7 |
| Phase | Phase-3 per-story adversarial convergence (BC-5.39.001), Wave-5, found on a fresh-context lens after 4+ rounds |
(Related theme: a prior local candidate on tautological-test detection at the TDD-RG gate — same family, distinct mechanism: that one is about vacuous assertions, this one is about a correct assertion under a contradictory name + contradictory spec prose.)
Contributor guide
Assessment
This issue has not been assessed yet.