BOHICA-LABS / BOHICA-LABS/vsdd-factory

process-gap(adversary+orchestrator): spec-convergence loop can 'converge' on story delivery scope that was never shipped — no pre-verdict grep gate for declared delivery surface

Open
#440 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2
Forks
1
Avg merge
6h 43m
Merged PRs (30d)
29

Description

## Symptom

STORY-014 (config plumbing + AutoRejectTimer daemon wiring, akey) ran 10 spec-text iterations (v1.11 → v1.20) through the standard Phase-3 adversary convergence loop. Every iteration passed convergence relative to the immediately prior spec-text state, and 7 consecutive \"partial-fix regressions\" (S-7.01) were logged along the way. Pass #1 of v1.20 finally flagged the actual defect: **none of the story's declared delivery scope existed in the codebase**. Verified via:

```
grep -nE \"load_config|load_config_strict|apply_env_overrides|expand_tildes|literal_defaults|load_config_from_toml|AutoRejectTimer\" \\
crates/akey/src/{pipeline,main}.rs crates/akey/src/cli/mod.rs
# zero hits
```

10 versions of spec convergence had been grading spec-text mutations against spec-text mutations. The shipped-code state was invisible to the loop.

## Distinct from existing issues

- **#399 (phantom-API drift)** — specs cite symbols absent from *partially* shipped code. This is worse: the *entire* delivery scope is absent and the loop still 'converges'.
- **#322 (remediation diffs get no targeted re-review)** — sibling; that issue covers regressions in *shipped* remediations. This one covers convergence on the *absence* of any shipment.
- **#419 (impl adds public API not documented in api-surface)** — reverse direction; that's spec missing surface. This is surface missing impl.

## Root cause

`vsdd-factory:adversary` dispatch for a story-scoped pass has no mandatory pre-verdict gate that asserts:

> Every symbol, module, function, wiring point, and CLI flag named in the story's `## Deliverables` / `## Files to Create/Modify` / `## Acceptance Criteria` sections MUST be greppable in the target repo before the adversary is allowed to issue a CLEAN verdict.

Absent this gate, the adversary reads the story + specs + prior adversary transcripts, finds internally-consistent spec text, and issues CLEAN — regardless of whether the code ever landed. The orchestrator likewise trusts the CLEAN verdict without an independent presence check.

## Downstream cost

- STORY-014 wasted 10 convergence iterations on spec-text-vs-spec-text remediation.
- The 7-consecutive partial-fix regression streak (S-7.01) was a *symptom*, not the root cause — the adversary was correctly flagging spec-text drift while blind to the missing shipment.
- Task #33 (\"config plumbing complete\") had been marked complete earlier without the delivery scope landing, and no downstream gate caught it.

## Proposed remediation

**Adversary pass preflight (mandatory when the story declares a delivery scope):**

1. Parse story frontmatter / body for declared symbols, function names, modules, CLI flags, and file paths under `Deliverables` / `Files` / `Acceptance Criteria`.
2. Run `grep -nE '||...'` against the declared target files.
3. If **any** declared symbol is absent: emit a `NOT CLEAN (delivery-scope-missing)` verdict pre-empting the spec-text review, with the missing-symbol list attached. Do NOT proceed to spec-text convergence.
4. Only proceed to normal adversary review once the presence gate passes.

**Orchestrator dispatch check (belt-and-braces):**

Before dispatching adversary for a story that has iterated ≥3 spec versions, orchestrator runs the same grep gate and refuses dispatch if delivery-scope-missing — routes to stub-architect + implementer instead.

## Repro / evidence

- akey `.factory/stories/STORY-014-config-parsing-daemon-wiring.md` v1.11 → v1.20 changelog rows.
- 7 partial-fix regression entries in the adversary transcripts for those passes.
- Grep verification above.
- Delivery ultimately shipped as PR drbothen/akey#34 (squash `a3fbf04`) via a manual stub-architect → test-writer → implementer → pr-manager burst after the loop was broken.

## Related

- #322 (remediation not re-reviewed)
- #399 (phantom-API drift — narrow case of this)
- #419 (impl adds undocumented surface — reverse case)
- #289 / #299 / #332 (surface-set mismatch class)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.