BOHICA-LABS / BOHICA-LABS/vsdd-factory

bug(convergence): only machine-checked predicates are a round count and CRIT/HIGH — MEDIUMs do not break the streak, coverage is never checked, and monotonicity penalizes fixing it

Open
#820 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2
Forks
1
Avg merge
6h 43m
Merged PRs (30d)
29

Description

## Summary

Three separate problems in the convergence gate, all readable in the shipped source. Together they let a story reach "3 consecutive clean passes" without any guarantee that the artifact was read, while actively discouraging the one fix that would close the gap.

## 1. MEDIUM and LOW findings do not break the clean streak

`hooks/convergence-tracker.sh`, the streak loop (around lines 141–167) reads exactly two rows per pass file:

```sh
PF_CRIT=$(grep -i "| *CRITICAL *|" "$pf" ...)
PF_HIGH=$(grep -i "| *HIGH *|" "$pf" ...)
...
if [[ "$PF_CRIT" -eq 0 ]] && [[ "$PF_HIGH" -eq 0 ]]; then
CLEAN_STREAK=$((CLEAN_STREAK + 1))
```

A pass reporting five MEDIUM findings and zero CRITICAL/HIGH increments `CLEAN_STREAK`. Three such passes satisfy the gate. The skill text calls the streak "clean passes," which reads as *no findings*, and the hook's arithmetic means *no CRITICAL or HIGH findings*.

Either is defensible as a policy. They should not disagree with each other, and right now the machine-checked one is the looser one.

## 2. There is no coverage predicate anywhere

The gate's complete set of machine-checked conditions is: 0 CRITICAL in this pass, 0 HIGH in this pass, `CLEAN_STREAK >= 3`. Nothing requires that any particular part of the artifact under review has been read by any pass.

The engine already knows this is a live risk. `skills/adversarial-review/SKILL.md:202`:

> the adversary makes genuinely new findings through pass 9+ in complex projects, including findings every prior pass missed (e.g., phantom crate references that only surface when the adversary reads dependency-graph.md with truly fresh eyes)

That is a statement that *what got read* is the variable that matters. The gate does not measure it.

The failure has a mechanism, not just a probability. Fresh reviewers are drawn to the region with the densest citation anchors — which is the most-corrected region, because corrections add citations. So successive independent passes converge on the same neighbourhood and leave the same regions unopened, and each pass is genuinely fresh while the *union* of passes is not.

What that produced here: ten rounds on one story. Round ten opened a design region no earlier round had opened with code in hand and returned four MAJOR findings, one of which was that a mandated code construct does not compile (filed separately). Three of the later rounds each found defects that would otherwise have shipped. My estimate of how much of the design document had been read by *anyone* across all ten rounds is 35–45% — that is an estimate from pass records, not a measurement, and I flag it as such.

## 3. The monotonicity rule penalizes the fix

`skills/adversarial-review/SKILL.md`, Trajectory Monotonicity:

> Finding counts must decrease monotonically across passes. If any pass shows MORE findings than the previous pass, this is a regression — stop and investigate root cause before proceeding.

Deliberately widening the reviewed surface raises the finding count. That is the expected and desired result of fixing problem 2 — and the engine classifies it as a regression requiring a stop. The listed causes ("new scope was added without pre-validation", "the adversary's perimeter expanded unexpectedly") frame perimeter growth as a defect.

As written, an operator cannot both expand coverage and satisfy monotonicity. I hit this directly: the round that found the most was the round that read the most, and by the letter of the rule that round is a regression.

## Asks

1. **Make the streak definition and the hook agree.** Either count MEDIUM as streak-breaking, or state plainly in the skill that "clean" means zero CRITICAL and zero HIGH and that MEDIUMs may accumulate across a converged story.
2. **Add a coverage ledger to the gate.** The shape that worked here, after ten rounds demonstrated the round count wasn't doing the job: the streak may not *start* until every section of the artifact under review has been read by some round, tracked across rounds rather than within one. It converts "we ran enough passes" into "the document was read," which is the property the gate is trying to buy.
3. **Exempt deliberate scope expansion from the monotonicity regression rule**, or the rules are jointly unsatisfiable. A finding-count rise attributable to a declared coverage expansion is evidence the expansion was worth doing.

Observed in `run/switchboard-blue` during S-BL.ACCESS-CONNECTOR Step-4.5 convergence, plugin `1.0.0-rc.24`.

Contributor guide

Open the contributing guide

Research direction

The issue points to hooks/convergence-tracker.sh (lines 141–167) for the streak logic and skills/adversarial-review/SKILL.md for the monotonicity rule. Start by reading those files to understand the current gate conditions. To test changes, you'll need to understand how pass files are generated and how the streak is tracked. 'Done' means the three problems are addressed: streak definition matches policy, a coverage ledger is added, and monotonicity allows deliberate scope expansion.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, shell
Domain
backend-api-design, devtools, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.