BOHICA-LABS / BOHICA-LABS/vsdd-factory

process-gap(adversary+implementer): comment-truth defects survive spot-fix passes — remedy is disposition sweep, not per-finding patch

Open
#484 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2
Forks
1
Avg merge
6h 43m
Merged PRs (30d)
29

Description

## Summary

When an adversarial pass finds a comment or docstring that asserts unimplemented, stub, or wrong behavior, a per-finding patch (\"fix this comment\") predictably fails to eliminate the class. The effective remedy is a grep-driven **disposition sweep** of the whole diff: every hit of stub/assertion vocabulary is either marked FIXED or ACCURATE-KEPT with justification, with no silent skips. Spot-fixing per finding continues to leave sibling instances that the next pass finds — restarting the convergence counter.

## Evidence (single story, private pilot; identifiers scrubbed)

Seven recurrences of comments/docstrings asserting unimplemented or wrong behavior on one P0 story, across a multi-pass adversarial loop:

| Pass | Findings found | How dispatched | Outcome |
|------|---------------|----------------|---------|
| Spot-fix pass 1 | 2 comment-truth findings | Fix these 2 | Resolved; 4 more found next pass |
| Spot-fix pass 2 | 4 comment-truth findings | Fix these 4 | Resolved; 3 more found next pass |
| Disposition sweep | 3 remaining + grepped corpus | grep for stub vocabulary → disposition every hit | Class eliminated from diff |

The vocabulary targeted by the grep: `assert(false`, `stub`, markers of unreached state, phrases documenting design-intent rather than implemented behavior. Every corpus hit was dispositioned FIXED or ACCURATE-KEPT; none were silently skipped.

Note: even the sweep pass initially missed one line, caught only by orchestrator line-level re-verification — indicating the sweep must be exhaustive, not best-effort.

## Root cause

Spot-fix dispatches are finding-scoped, not class-scoped. An adversary instructed to fix \"this comment\" cannot know whether sibling comments of the same class exist elsewhere in the diff. The comment-truth class is structurally a whole-diff property: any residue anywhere in the diff constitutes a violation.

## Proposed protocol change

**Trigger rule:** when an adversary pass returns ≥ 2 comment-truth findings in a single pass, the fix dispatch must mandate a disposition sweep of the whole diff — not per-finding patches.

**Sweep protocol:**
1. Grep the diff for stub/assertion vocabulary (e.g., `assert(false`, `STUB`, `RED-GATE`, `GREEN-BY-DESIGN`, `TODO`, names of unreached states).
2. For every hit: disposition as FIXED (comment removed or corrected to match implementation) or ACCURATE-KEPT (with justification: the comment remains correct, and here is why).
3. No silent skips. The disposition list is emitted as part of the fix report.
4. Orchestrator re-verifies the disposition list against the committed diff before accepting.

The ≥2-finding trigger is conservative; a sweep-first policy on any comment-truth finding would be even stronger.

## Relationship to existing issues

- **#382 (stale/overclaiming doc-comments):** #382 concerns behavioral-claim staleness (exhaustiveness, panic-freedom claims that rot as code changes) with an adversary-mandate addition. This issue concerns the *remediation dispatch strategy*: per-finding vs. whole-diff sweep. Adjacent root class; distinct failure mechanism and remedy.
- **#288 (stale stub docstrings surviving post-impl):** #288 is about stub-time citations lingering after the implementer fills the body. This issue is about stub/assertion vocabulary surviving adversarial passes that were specifically dispatched to fix them. The survival mode differs (implementer oversight vs. spot-fix dispatch strategy).
- **#475 (stub-architect test-file authorship):** cross-ref — the same stub-vocabulary artifacts that #475 addresses at stub-gate are a subset of the comment-truth corpus this sweep would cover post-impl.

*(From a private pilot repo; framework-pattern detail only.)*

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.