BOHICA-LABS / BOHICA-LABS/vsdd-factory
adversary: corroboration-inversion — fresh-context findings quote evidence that contradicts actual file contents
- Dominant language
- Rust
- Stars
- 2
- Forks
- 1
- Avg merge
- 6h 43m
- Merged PRs (30d)
- 29
Description
## Summary
Fresh-context adversary sub-agent can produce high-severity findings whose cited evidence is INVERTED from the actual source. This is not phantom-symbol drift (#399) — this is phantom-evidence drift: the symbol exists at the cited file:line, but the adversary's quoted content contradicts what is on disk. The adversary asserts "line X does A" when line X verifiably does the opposite of A.
## Observed pattern (abstract)
A well-formed story with a recently-ratified BC (audit-ordering / CAS-first / fail-closed contract) was implemented correctly and had all integration tests passing green. On fresh-context Pass-1:
- Adversary returned 15 findings (4 CRIT / 5 HIGH / 5 MED / 3 LOW).
- On triage against actual HEAD source: **7 of 15 findings (~47%) were inverted-evidence hallucinations.**
- Specifically:
- Finding cites 'line N calls method A' → line N calls sibling method B (with different semantics that exactly satisfy the invariant the finding claims is violated).
- Finding claims 'doc-comment declares ordering X' → doc-comment declares ordering Y verbatim.
- Finding declares 'test WILL FAIL' → target test passes green in immediate re-run.
- Remaining 8 findings included 2 legitimate doc-hygiene issues, 1 legitimate scope question needing product-owner ruling, and 5 minor observability improvements.
## Distinct from related issues
- **#399 (phantom-API drift):** spec cites a symbol that doesn't exist → adversary passes CLEAN. This variant: symbol exists and code is correct → adversary invents contradictory evidence.
- **#405 (adversary asked about details out of scope):** adversary invited to hallucinate by prompt scope. This variant: prompt scope is correct; adversary hallucinates against the correct target.
- **#18-equivalent (adversary stalls on large-context corroboration):** adversary fails to corroborate. This variant: adversary produces confident anti-corroboration.
## Suggested remediation
Two layers, cheap:
### 1. Structured-finding schema
Require adversary to emit each finding in structured shape:
\`\`\`json
{
\"severity\": \"CRIT\" | \"HIGH\" | \"MED\" | \"LOW\",
\"file\": \"crates/foo/src/bar.rs\",
\"line\": 293,
\"evidence_quote_verbatim\": \"self.queue.remove(request.id).await;\",
\"invariant_violated\": \"BC-N.M.P POST-Q ...\",
\"suggested_fix\": \"...\"
}
\`\`\`
The \`evidence_quote_verbatim\` must be a byte-exact substring of the file (no paraphrase). System prompt tells the adversary this is machine-verified.
### 2. Post-adversary evidence-verification pass
Before dispatching remediation, orchestrator (or lightweight validator sub-agent) executes for every finding with severity ≥ HIGH:
\`\`\`
Read(file, line-2..line+5)
assert evidence_quote_verbatim in window
\`\`\`
Findings failing verification are tagged HALLUCINATED and NOT dispatched to implementer. They're surfaced to the human orchestrator for manual triage.
## Impact / downstream cost
- Convergence counter poisoning: false CRIT/HIGH findings reset 3-CLEAN to 0/3, burning subsequent adversary passes on findings that can't be remediated (because the code already does the right thing).
- Wasted implementer bursts: if orchestrator dispatches remediation without triage, implementer either reports 'no change needed' (best case) or, worse, is nudged into REMOVING correct code because the finding's rationale sounds plausible.
- Manual grep-triage: orchestrator has to Read every cited file:line by hand to distinguish real findings from hallucinated ones. At project cadence (~1-2 stories/day, 3+ fresh passes per story) this is on the order of 30-90 minutes per hallucination event. Multiplied across a dozen-story wave, aggregate cost is high.
## Why fresh-context makes this WORSE, not better
Fresh-context is supposed to reduce hallucination by starving the model of contaminating priors. But when the target artifact is complex (multi-hundred-line src file with intricate ordering invariants), the fresh model has to construct its mental model from a cold read. Under cognitive load it can:
- Read line X, form initial hypothesis about ordering.
- Continue reading and update the model.
- When emitting the finding, retrieve the initial hypothesis rather than the updated one → cite line X's actual content as evidence for the OPPOSITE claim.
The structured-schema + post-verification approach above closes this failure mode without requiring model-level fixes.
## Willing to share
Anonymized excerpts of the 7 hallucinated findings + corresponding actual source line contents available on request — useful as calibration data for prompt engineering or eval harness.
Contributor guide
Assessment
This issue has not been assessed yet.