BOHICA-LABS / BOHICA-LABS/vsdd-factory

process-gap(implementer+adversary): doc-comments asserting behavior are not verified against the code they document; stale/overclaiming doc-comments survive all gates

Open
#382 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2
Forks
1
Avg merge
6h 43m
Merged PRs (30d)
29

Description

## Class

A function/module doc-comment states a behavioral claim (\"returns X when Y\", \"never panics on Z\", \"exhaustive over all variants of E\") that is not checked against the actual code. When the implementation later changes, or when the doc-comment overstated from the start, the divergence survives every gate:

- The Red/Green TDD cycle tests *behavior*, not *documentation*, so a wrong doc-comment never fails a test.
- clippy/rustfmt (or equivalent) do not validate semantic claims in prose.
- The adversary reads code and specs but is not mandated to diff doc-comment claims against the code they annotate.
- Consistency validation is scoped to cross-artifact references, not intra-file prose-vs-code.

Result: doc-comments become a second, unverified spec surface that drifts silently from behavior.

## Observed shape (abstracted)

A doc-comment claimed a mapping function was \"exhaustive over all variants\" of an enum. The claim was true at write time but is exactly the kind of statement that rots when a variant is added — nothing in the pipeline would catch the doc-comment going stale, because the match itself would still compile (with a catch-all) or fail-compile (without one) independent of the prose.

Separately, a doc-comment described a truncation helper as \"safe\" without the test suite ever exercising the unsafe input class the word \"safe\" implicitly claims coverage of.

## Proposed mitigation (engine-level)

1. Adversary mandate addition: for any doc-comment that makes a *verifiable behavioral claim* (exhaustiveness, panic-freedom, totality, ordering, byte/charset safety), the adversary must either (a) locate the test that discharges the claim, or (b) flag the claim as unverified.
2. A lightweight lint/heuristic that flags doc-comment keywords implying total properties (\"exhaustive\", \"never panics\", \"always\", \"safe for all\", \"total\") on functions lacking a linked property test or VP citation.
3. Guidance to implementer/test-writer: a behavioral doc-comment is a claim that must be discharged by a test or explicitly marked as informal.

## Relationship to existing issues

Distinct from #367 (stale story-id pointers — rename residual) and #333 (false upstream dependency claims in stories). Those concern *reference* staleness; this concerns *behavioral* claims in code prose diverging from the code's actual behavior. Cousin of #381 (false-independence oracle) in that both are unverified claims that look like verification.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.