BOHICA-LABS / BOHICA-LABS/vsdd-factory
process-gap(implementer+adversary): doc-comments asserting behavior are not verified against the code they document; stale/overclaiming doc-comments survive all gates
- Dominant language
- Rust
- Stars
- 2
- Forks
- 1
- Avg merge
- 6h 43m
- Merged PRs (30d)
- 29
Description
## Class
A function/module doc-comment states a behavioral claim (\"returns X when Y\", \"never panics on Z\", \"exhaustive over all variants of E\") that is not checked against the actual code. When the implementation later changes, or when the doc-comment overstated from the start, the divergence survives every gate:
- The Red/Green TDD cycle tests *behavior*, not *documentation*, so a wrong doc-comment never fails a test.
- clippy/rustfmt (or equivalent) do not validate semantic claims in prose.
- The adversary reads code and specs but is not mandated to diff doc-comment claims against the code they annotate.
- Consistency validation is scoped to cross-artifact references, not intra-file prose-vs-code.
Result: doc-comments become a second, unverified spec surface that drifts silently from behavior.
## Observed shape (abstracted)
A doc-comment claimed a mapping function was \"exhaustive over all variants\" of an enum. The claim was true at write time but is exactly the kind of statement that rots when a variant is added — nothing in the pipeline would catch the doc-comment going stale, because the match itself would still compile (with a catch-all) or fail-compile (without one) independent of the prose.
Separately, a doc-comment described a truncation helper as \"safe\" without the test suite ever exercising the unsafe input class the word \"safe\" implicitly claims coverage of.
## Proposed mitigation (engine-level)
1. Adversary mandate addition: for any doc-comment that makes a *verifiable behavioral claim* (exhaustiveness, panic-freedom, totality, ordering, byte/charset safety), the adversary must either (a) locate the test that discharges the claim, or (b) flag the claim as unverified.
2. A lightweight lint/heuristic that flags doc-comment keywords implying total properties (\"exhaustive\", \"never panics\", \"always\", \"safe for all\", \"total\") on functions lacking a linked property test or VP citation.
3. Guidance to implementer/test-writer: a behavioral doc-comment is a claim that must be discharged by a test or explicitly marked as informal.
## Relationship to existing issues
Distinct from #367 (stale story-id pointers — rename residual) and #333 (false upstream dependency claims in stories). Those concern *reference* staleness; this concerns *behavioral* claims in code prose diverging from the code's actual behavior. Cousin of #381 (false-independence oracle) in that both are unverified claims that look like verification.
Contributor guide
Assessment
This issue has not been assessed yet.