BOHICA-LABS / BOHICA-LABS/vsdd-factory
enhancement(adversarial-loop): hardening sweep — pending-reason format + convergence metrics + mid-loop state + implementer scope
- Dominant language
- Rust
- Stars
- 2
- Forks
- 1
- Avg merge
- 6h 43m
- Merged PRs (30d)
- 29
Description
## Summary
Four related quality-of-life gaps observed during a long adversarial convergence loop (9 passes on a single story). Each is small individually; together they make the loop harder to operate, recover, and audit.
## Gap 1 — `pending()` reason format inconsistent
Test-writers produce `pending()` calls with free-form reasons:
- "See BC-12.12.001" (BC-anchored deferral)
- "Phase 4 holdout" (integration deferral)
- "See known-issues.md for deferral" (sometimes points at non-existent entries — observed once)
- (no reason at all in some cases)
Adversaries can't mechanically validate breadcrumbs because the format is free-form. One adversary pass found a `pending()` that cited a non-existent KI entry — the breadcrumb was broken and no automated check could have caught it.
**Suggested format:**
```gdscript
pending("DEFER: see ")
# class in {cross-bc, integration-holdout, scene-tree, hardware, spec-amend}
# ref = BC-ID | KI-ID | STORY-ID | "phase-4"
```
Adversary can then grep + validate references mechanically.
## Gap 2 — Convergence trajectory observability
Orchestrators currently judge "continue or stop" on intuition. Findings-per-pass count, token-spend, and predicted convergence pass are not surfaced in any structured way. Decisions like "this story is stalling" or "this is on the convergence path" rely on manual review of agent reports.
**Suggested artifact:** orchestrator emits a `convergence-trajectory.md` table per story:
| Pass | Result | CRIT | MAJOR | MINOR | NITPICK | Tokens (est) | Clean? |
|------|--------|------|-------|-------|---------|--------------|--------|
| 1 | findings | 4 | 7 | 5 | 2 | 95k | no |
| 2 | findings | 2 | 5 | 6 | 2 | 88k | no |
| ... | ... | ... | ... | ... | ... | ... | ... |
This is easy to construct from agent reports + token-counters. Surfaces "stalling" stories early.
## Gap 3 — Adversary convergence-state mid-loop persistence
`adversary-convergence-state.json` is written AFTER convergence by state-manager at cycle close. During the loop, the orchestrator has no persistent state — if the session dies (API error, OOM, network blip), recovery means re-reading agent reports from transcripts.
**Suggested fix:** state-manager appends each pass result to the convergence-state.json incrementally. The orchestrator can resume mid-loop by reading the file's current state. Auto-recovery via the existing `/vsdd-factory:recover-state` skill becomes feasible.
## Gap 4 — Implementer scope-creep on test files
Implementers regularly modify test files mid-implementation:
- Sometimes legitimately (fixing test-tool API misuse — see related issue on GUT signal-arg trap)
- Sometimes scope-creep (changing assertions, adding tests, deleting tautological tests)
Currently ad hoc: implementer self-judges. There's no protocol for "is this a tool-fix or a content-fix" or an audit trail of which test changes were made by implementer vs test-writer.
**Suggested protocol:** implementer may modify tests ONLY for test-tool-correctness fixes (API misuse, syntax issues, harness compatibility). MUST list each modification in the exit report under a "Test-file modifications" section. Orchestrator escalates non-tool fixes back to test-writer for proper handling.
## Why these belong together
All four gaps are about *operating a multi-round adversarial convergence loop with confidence*. Each individually is a small fix. Together they make the loop auditable, recoverable, and predictable. Splitting them into 4 separate issues would create noise; combining them into a single hardening tracker matches the actual operational pattern.
## Related
- #258 (orchestrator mis-routes synthesized interrupts) — sibling operational pain
- #261 (privacy meta) — relevant for sanitization
- #267 (state-manager source-of-truth validation) — overlapping concern in Gap 1 (breadcrumb validity)
Contributor guide
Assessment
This issue has not been assessed yet.