BOHICA-LABS / BOHICA-LABS/vsdd-factory
process-gap(adversary+implementer): comment-truth defects survive spot-fix passes — remedy is disposition sweep, not per-finding patch
- Dominant language
- Rust
- Stars
- 2
- Forks
- 1
- Avg merge
- 6h 43m
- Merged PRs (30d)
- 29
Description
## Summary
When an adversarial pass finds a comment or docstring that asserts unimplemented, stub, or wrong behavior, a per-finding patch (\"fix this comment\") predictably fails to eliminate the class. The effective remedy is a grep-driven **disposition sweep** of the whole diff: every hit of stub/assertion vocabulary is either marked FIXED or ACCURATE-KEPT with justification, with no silent skips. Spot-fixing per finding continues to leave sibling instances that the next pass finds — restarting the convergence counter.
## Evidence (single story, private pilot; identifiers scrubbed)
Seven recurrences of comments/docstrings asserting unimplemented or wrong behavior on one P0 story, across a multi-pass adversarial loop:
| Pass | Findings found | How dispatched | Outcome |
|------|---------------|----------------|---------|
| Spot-fix pass 1 | 2 comment-truth findings | Fix these 2 | Resolved; 4 more found next pass |
| Spot-fix pass 2 | 4 comment-truth findings | Fix these 4 | Resolved; 3 more found next pass |
| Disposition sweep | 3 remaining + grepped corpus | grep for stub vocabulary → disposition every hit | Class eliminated from diff |
The vocabulary targeted by the grep: `assert(false`, `stub`, markers of unreached state, phrases documenting design-intent rather than implemented behavior. Every corpus hit was dispositioned FIXED or ACCURATE-KEPT; none were silently skipped.
Note: even the sweep pass initially missed one line, caught only by orchestrator line-level re-verification — indicating the sweep must be exhaustive, not best-effort.
## Root cause
Spot-fix dispatches are finding-scoped, not class-scoped. An adversary instructed to fix \"this comment\" cannot know whether sibling comments of the same class exist elsewhere in the diff. The comment-truth class is structurally a whole-diff property: any residue anywhere in the diff constitutes a violation.
## Proposed protocol change
**Trigger rule:** when an adversary pass returns ≥ 2 comment-truth findings in a single pass, the fix dispatch must mandate a disposition sweep of the whole diff — not per-finding patches.
**Sweep protocol:**
1. Grep the diff for stub/assertion vocabulary (e.g., `assert(false`, `STUB`, `RED-GATE`, `GREEN-BY-DESIGN`, `TODO`, names of unreached states).
2. For every hit: disposition as FIXED (comment removed or corrected to match implementation) or ACCURATE-KEPT (with justification: the comment remains correct, and here is why).
3. No silent skips. The disposition list is emitted as part of the fix report.
4. Orchestrator re-verifies the disposition list against the committed diff before accepting.
The ≥2-finding trigger is conservative; a sweep-first policy on any comment-truth finding would be even stronger.
## Relationship to existing issues
- **#382 (stale/overclaiming doc-comments):** #382 concerns behavioral-claim staleness (exhaustiveness, panic-freedom claims that rot as code changes) with an adversary-mandate addition. This issue concerns the *remediation dispatch strategy*: per-finding vs. whole-diff sweep. Adjacent root class; distinct failure mechanism and remedy.
- **#288 (stale stub docstrings surviving post-impl):** #288 is about stub-time citations lingering after the implementer fills the body. This issue is about stub/assertion vocabulary surviving adversarial passes that were specifically dispatched to fix them. The survival mode differs (implementer oversight vs. spot-fix dispatch strategy).
- **#475 (stub-architect test-file authorship):** cross-ref — the same stub-vocabulary artifacts that #475 addresses at stub-gate are a subset of the comment-truth corpus this sweep would cover post-impl.
*(From a private pilot repo; framework-pattern detail only.)*
Contributor guide
Assessment
This issue has not been assessed yet.