BOHICA-LABS / BOHICA-LABS/vsdd-factory
process-gap(orchestrator+adversary): remediation diffs get no deterministic targeted re-review — regressions caught only probabilistically by the next full pass
- Dominant language
- Rust
- Stars
- 2
- Forks
- 1
- Avg merge
- 6h 43m
- Merged PRs (30d)
- 29
Description
## Class of failure
The convergence loop's only check on a **remediation diff** is the *next* full fresh-context adversary/consistency-validator pass. That pass is a whole-corpus review, not a targeted re-review of the lines just changed. So a regression introduced *by the remediation itself* is caught only **probabilistically** — it depends on whether the next stochastic full pass happens to traverse those exact surfaces before it emits a verdict (and, per #306, before it compacts).
This is the structural reason a convergence streak can ship **two consecutive partial-fix passes**: pass N remediates defect X but introduces stale-binding Y; pass N+1 (fresh context, full corpus) happens to find Y and resets the streak; pass N+1's remediation of Y introduces Z; and so on. Each individual pass is "clean by its own lights"; the loop converges slower than the pass count suggests, and the gate's reliability is a function of review luck, not a deterministic property.
Relationship to existing issues (this is the gap *between* them, not a dup):
- #299 proposes a guard for one specific defect class (duplicated-value set-equality). Necessary but per-class.
- #277 / #310 cover derived-count drift; #309 covers lint-class gates invisible to ground truth. All are *specific* surfaces.
- #283 Gap 2 surfaces convergence *trajectory observability* (continue/stop intuition) — measurement, not a gate.
- #222 documents the *rationale* for "3 clean passes"; this issue is about what "clean" is actually testing.
None of them make the **remediation diff itself** a first-class review object.
## Suggested direction
Add a deterministic, cheap **post-remediation self-check step** that runs *before* the next full adversarial pass, scoped to what the remediation touched:
1. **Diff-scoped binding sweep.** For every identifier re-anchored in the remediation (`X: A → B`), grep the whole corpus for residual `X → A` and fail if any survive (excluding immutable changelog prose). Generalizes #299's guard to "whatever this diff just changed."
2. **Diff-scoped derived-field recompute check.** If the diff added/removed primary content (BC, VP, story), assert the relevant counts/totals were recomputed (the #277/#310 surfaces) — but triggered *by the diff*, not by a separate validator pass.
3. **Surface the result as gate input.** The orchestrator attaches this self-check's output to the next adversary prompt as ground truth (same mechanism #298/#309 propose for the test/lint tally).
The point is not a new heavyweight reviewer — it's making the remediation step *deterministically self-verifying for the class of regressions remediations actually introduce*, so the next full pass is confirming convergence rather than being the first and only thing that might catch a self-inflicted regression.
## Environment
- Plugin: vsdd-factory `1.0.0-rc.21`
- Observed: orchestrator-driven Phase-1 spec convergence, fresh-context `adversary` + `consistency-validator` pair per pass, "3 consecutive clean passes" gate. Two consecutive passes each shipped (and the following pass caught) a partial-fix regression in spec cross-references.
(No private project content included, per #261.)
Contributor guide
Assessment
This issue has not been assessed yet.