BOHICA-LABS / BOHICA-LABS/vsdd-factory

process-gap(orchestrator+adversary): N-consecutive-clean convergence counts finding-absence with no vector-coverage-novelty requirement — a defect in an unprobed vector survives arbitrarily many 'clean' passes if fresh-context lenses re-probe the same vectors

Open
#710 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2
Forks
1
Avg merge
6h 43m
Merged PRs (30d)
29

Description

## Summary

The per-story adversarial convergence gate (BC-5.39.001: *N* consecutive clean fresh-context passes, any finding resets to 0) defines a "clean pass" purely as **finding-absence**. There is no measure of whether a given pass explored **new** vectors versus re-confirming vectors already probed by earlier passes. Consequently, a real defect residing in a never-probed vector can survive an unbounded number of "clean" passes whenever successive fresh-context adversary instances gravitate to the same vectors that the story/BC framing makes salient.

"Clean" is being read by the orchestrator (and by the convergence counter) as "fully probed." It actually means "the vectors this pass happened to look at contained no findings." Those are not the same claim, and the gap between them is invisible at the counter level.

## Concrete instance (akey STORY-024, audit_fail_mode fail-closed audit path)

Finding-severity trajectory across passes:
- Pass 1: MEDIUM code + LOWs (real code defects, remediated)
- Pass 2: MAJOR incomplete-remediation + LOWs (remediated)
- Pass 3: 2 LOW **doc-only** (mechanism comment, phantom citation)
- Pass 4: 2 LOW **doc-only** (version drift, line-number drift)
- Pass 5: **2 MEDIUM code/concurrency defects** — a DI-004 silent audit-drop on the signing hot path (Written-path `append()` racing recreation, unserialized) + an EC-005/EC-006 violation on the `append_identities_listed` path (no mode-aware disappearance dispatch)

Passes 3 and 4 were "code-clean" — the only findings were cosmetic. A naive reading says the code was converging. In reality the code carried two MEDIUM defects the whole time; passes 1–4 simply never constructed the interleaving that exposes them. The two acceptance tests covering the relevant BC clause both `remove_file()` **before** launching their racing tasks, funneling every prior concurrency look into one branch. Pass 5 found the defects only because a fresh lens reasoned about a different interleaving on its own initiative — not because the protocol required new coverage.

Had the story reached 2/3 or 3/3 clean on the strength of passes 3+4 (doc-clean) plus one more doc-clean pass, the gate would have declared convergence with two MEDIUM signing-path defects live.

## Why this is distinct from adjacent issues

- **#700** (test-writer+adversary) is the *test-artifact* root cause: within a single delivery, an AC's gate exercises a subset of the paths its obligation names. That is the mechanism by which a vector goes unprobed. **This issue** is about the *convergence-counter semantics*: the gate increments toward termination on finding-absence with no coverage-novelty gate, so #700-class gaps (and any other unprobed-vector defect) are not merely possible but *undetectable at the convergence layer*. #700 is the leak; this is the gauge that reads "full" anyway.
- **#344** (hygiene residual stream fails to terminate) is the *opposite* failure: cosmetic findings keep resetting a counter that *should* terminate. This issue is a counter that terminates (or would) when it *should not* — real defects hidden behind vector-repetition.

## Suggested direction (not prescriptive)

- Track, per pass, *which vectors/lenses/code-paths were exercised* (even coarsely — e.g. the adversary declares "vectors probed: {…}" in its verdict). A pass that adds no new vector coverage over the prior clean pass should not, by itself, advance the convergence count — or should at minimum be flagged so the orchestrator can direct the next pass at unprobed territory.
- Alternatively/additionally: derive a coverage checklist from each obligation's *enumerated* code paths (ties into #700) and require the convergence gate to confirm every enumerated path has at least one constructing test/lens before "clean" counts toward termination.
- Make the orchestrator's per-pass dispatch coverage-aware: rotate/assign lenses to *under-probed* vectors rather than re-running identical briefs, so successive passes diversify by construction.

## Severity

Process-gap, medium. The "any finding resets" + fresh-context + parallel-diverse-lens design is a *probabilistic* guarantee that enough independent looks eventually hit every vector; it worked here (Pass 5 caught it). The defect is that the guarantee is probabilistic with no coverage floor, and the convergence counter treats a vector-repeating clean pass identically to a coverage-expanding one — so the operator's confidence at "3/3 clean" is not backed by any completeness measure. For security-critical fail-closed paths (this instance: silent audit drop enabling a signature with no durable record), that false-confidence window is the risk.

Reported from an akey (ArcavenAE, private) STORY-024 convergence run; happy to supply the full pass-by-pass finding ledger if useful.

Contributor guide

Open the contributing guide

Research direction

The issue describes a process gap in an adversarial testing framework. Start by understanding the orchestrator and adversary components, likely in the `src/` directory. Look for the convergence counter logic and how a 'clean pass' is defined. Examine the test artifacts and pass ledgers from STORY-024 to see the vector coverage problem. Determine what changes to the pass-tracking or dispatch logic would introduce coverage-novelty requirements.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.