BOHICA-LABS / BOHICA-LABS/vsdd-factory
bug(orchestrator): adversary L1 task prompt asks about details out of scope for opaque-string plumbing story — invites hallucinated findings
- Dominant language
- Rust
- Stars
- 2
- Forks
- 1
- Avg merge
- 6h 43m
- Merged PRs (30d)
- 29
Description
## Summary
When orchestrator dispatches adversary L1 (impl-correctness) for a story whose scope is opaque-string plumbing (e.g., \"pass this address through without interpretation\"), a boilerplate L1 prompt that asks about \"hashing / derivation / endianness / normalization\" invites the adversary to invent findings about behavior the story explicitly disclaims.
## Concrete evidence (switchboard-blue Wave-6 Tranche B)
`S-BL.ROUTER-ADDR` scope: plumb a router address (opaque string) from config through PathTracker to Snapshot. No parsing, no hashing, no normalization — the story explicitly says \"opaque string.\"
Adversary L1 prompt across multiple passes asked about \"address parsing correctness / IPv6 handling / endianness of the derived key\" — none of which the story owns. This burned several passes on false-positive findings until Ruling-K clarified scope; and required test-writer to add comment obligations pinning the design (RULING-W6TB-K F-P4L2-03).
## Why it happens
Orchestrator's L1 dispatch prompt template is generic — it enumerates a broad L1 checklist. For \"transport-layer / plumbing\" stories the checklist is a superset of what applies. The adversary, having no way to know which checklist items are out-of-scope, dutifully investigates them and reports \"no evidence found for X\" as a HIGH finding.
## Proposed remediation
**Orchestrator dispatch:** derive an L1 sub-checklist from the story's declared scope tags. E.g., if `story.tags` includes `plumbing` or `opaque-transport`, drop \"parsing / normalization / derivation / endianness\" from the L1 checklist and add a leading clause: \"THIS STORY IS OPAQUE PLUMBING. Findings about the payload's internal structure are OUT OF SCOPE and must not be reported.\"
**Story template:** add a `scope_exclusions` frontmatter field listing behaviors the story explicitly does NOT own. Orchestrator injects these into every adversary dispatch.
**adversary policy:** any finding that names a behavior in `scope_exclusions` is auto-suppressed at ruling time.
## Cross-references
- #389 — preflight tuple accuracy (adjacent)
- #391 — mis-anchoring severity (this is the inverse: reviewer inventing an anchor)
- #278 — severity calibration drift (adjacent: this is calibration on scope, not severity)
## Severity
MEDIUM. Wastes passes and inflates finding counts; not a correctness defect but a discipline-erosion defect.
Contributor guide
Assessment
This issue has not been assessed yet.