BOHICA-LABS / BOHICA-LABS/vsdd-factory

process-gap(test-writer+story-writer): in-file 'AC deferred' banner in a test file with no owning follow-on story survives red-gate — implicit deferral without ticket

Open
#423 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2
Forks
1
Avg merge
6h 43m
Merged PRs (30d)
29

Description

## Process-gap

A test-writer can produce a red-gate test file whose header banner states, in prose, that a subset of the story's ACs are "deferred to a future task" — with no filed follow-on story, no update to the current story's frontmatter, and no marker the pipeline can grep for. The remaining tests satisfy compilability + red-gate mechanics, so orchestration accepts the burst and dispatches implementer. The deferred ACs are then implicitly out-of-scope for the current story but never explicitly re-scoped in the spec system.

Adversary Pass-1 catches it as a HIGH finding (deferred ACs with no owning story), but by then a full test-writer burst has been consumed and a scope-narrowing decision is required from product-owner before the story can proceed.

## Concrete shape (sanitized)

A story listed 8 acceptance criteria. The test-writer's output header contained the text (paraphrased): "AC #1–#5 covered here; AC #6–#8 deferred to a follow-on task as those ACs depend on a fixture not yet available." No follow-on story existed. The story frontmatter still listed all 8 ACs as in-scope. Red-gate mechanics passed. Adversary caught it.

## Why gates miss it

- Red-gate check counts failing tests, not tests-per-AC.
- Compilability + failure is asserted mechanically, not per-AC.
- test-writer's own report is prose; no structured artifact enumerates "AC-N: tested / deferred / skipped."
- No lint scans test files for the strings "deferred", "follow-on", "TODO", "skip" and cross-references AC-IDs.
- story-writer frontmatter is authoritative for AC scope; test-writer prose can silently contradict it.

## Distinct from adjacent issues

- **#369** (no lint on \`deferred:integration (STORY-NNN)\` pointer citations) — closest cousin, but that's *pointer-drift* on a properly-formatted marker; this is *creation of an ad-hoc, unstructured deferral in a test file with no marker to lint*. #369's proposed fix (grep for \`deferred:integration (STORY-NNN)\`) would not catch this because the deferral is prose.
- **#361** (BC EC narrowing/widening fix-bursts don't auto-dispatch story-writer for downstream sync) — different direction: that's *BC changes* driving *story sync*; this is a *test-writer* silently narrowing story scope.
- **#354** (BC-mandated test seams silently omitted) — different: that's stubs missing seams; this is tests missing whole ACs.

## Proposed remediation

1. **test-writer** dispatch checklist: emit a structured "AC-coverage manifest" per burst (JSON or YAML): \`[{ac: 'AC-01', status: 'tested', test_ids: [...]}, {ac: 'AC-06', status: 'deferred', reason: '...', owning_story: 'STORY-NNN'}]\`. If \`status: deferred\` and \`owning_story\` is null or unresolvable, orchestration REJECTS the burst.
2. **orchestrator/state-manager** gate: after every test-writer burst, verify AC coverage manifest matches story frontmatter. Any AC in frontmatter without a manifest entry (or with \`status: deferred\` and no owning story) fails the burst before red-gate check.
3. **adversary** policy: for each in-scope AC, verify at least one failing test names/exercises it. Deferral-by-prose is not deferral.
4. **story-writer** re-dispatch trigger: if a test-writer burst legitimately identifies a fixture blocker for a subset of ACs, the correct path is story-writer scope-narrowing + follow-on story creation *before* test-writer's output is accepted — not a banner in the test file.

## Impact

Every silent deferral is a spec/implementation drift. Even when adversary catches it, the cost is: one full test-writer burst wasted, one product-owner narrowing decision required, one story-writer re-dispatch, and one adversary re-run. Cheaper to reject at the burst boundary with a structured coverage manifest.

## Severity

MEDIUM. Recurs whenever a test-writer hits a fixture gap and chooses to narrow scope inline; needs a structural fix, not a policy exhortation.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.