BOHICA-LABS / BOHICA-LABS/vsdd-factory

feat(consistency-validator): add URL/endpoint/path coherence check across diagrams + tables + prose

Open
#131 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
2
Forks
1
Avg merge
6h 43m
Merged PRs (30d)
29

Description

## Summary

The `consistency-validator` agent / `validate-consistency` skill (vsdd-factory plugin) does not check URL / endpoint / path coherence across diagrams, tables, and body prose within the same spec package. This creates a class of substantive defects that survive multiple validation rounds even when the validator returns "clean."

## Reproduction (real case from `monocle` project, 2026-05-12)

The `monocle` project pre-Phase-1 spec package had three artifacts referencing the same wire-protocol endpoint with two different paths:

- **Vision document** (`domain-monocle-vision-synthesis.md` §Process Topology diagram): `POST .../hooks/user-prompt-submit`
- **Product brief** (`product-brief.md` §Scope): `POST /hooks/prompt-submit`
- **DTU assessment** (`dtu-assessment.md` clone scope): `/hooks/prompt-submit`

This divergence is a wire-contract bug: any implementer who reads the vision diagram and wires `/hooks/user-prompt-submit` builds a non-compliant daemon that fails clone validation.

The divergence survived **5 successive validation passes** (consistency-validator rounds 1-4 + validate-brief v3-v5) because the validator looks at:
- Identifiers (BC IDs, FR IDs, NFR IDs, OQ IDs)
- Anchor links (markdown `#section` references)
- Counts (number of crates, number of endpoints, MSRV versions)
- Naming (product name capitalization, term consistency)
- Cross-references (file paths in `inputs:`, `traces_to:`, `supplements:`)

It does NOT look at:
- URL paths in mermaid diagrams compared against URL paths in body prose tables
- Endpoint paths cited in prose vs. endpoint paths cited in schemas
- File paths embedded in body content (vs. frontmatter, which IS checked)

The case was caught only by a fresh-context adversarial pass (different agent type, different prompt heuristic).

## Proposed fix

Add a "URL / endpoint / path coherence" validation axis to the `consistency-validator` agent and the `validate-consistency` skill.

### Implementation sketch

1. **Extract URL/path candidates from all spec artifacts:**
- Mermaid code blocks (extract arrow targets like `--> POST /path`)
- Markdown tables (any cell containing `/` followed by alphanumeric)
- Code fences for any language that mentions paths (rust, yaml, toml, json)
- Body prose: regex `(?:GET|POST|PUT|DELETE|PATCH)\s+\S+/\S+` or just `/[a-z][a-z0-9-]+(?:/[a-z][a-z0-9-]+)*`

2. **Cluster candidates by semantic similarity:**
- Use edit distance + the surrounding context (within ~50 chars on each side) to group "this looks like the same endpoint"
- Example cluster: `/hooks/user-prompt-submit`, `/hooks/prompt-submit`, `POST .../hooks/user-prompt-submit` — all about UserPromptSubmit hook

3. **Flag clusters with multiple distinct paths:**
- For each cluster of N>1 candidates that share a semantic anchor but differ in path string, emit a finding:
```
URL coherence: 2 distinct path strings for same semantic endpoint
- /hooks/user-prompt-submit (vision-synthesis.md:66, line "POST .../hooks/user-prompt-submit")
- /hooks/prompt-submit (product-brief.md:98, dtu-assessment.md:100)
Resolution: choose canonical path and update all references.
```

4. **Integration with skill output:**
- Add a new section to the consistency report: "§URL/Path Coherence"
- Severity: BLOCKING (wire-contract divergence breaks implementations)

### Heuristic-only first cut (cheaper alternative)

If full clustering is too expensive, a simpler heuristic catches the most common pattern:
- Grep all spec artifacts for `/hooks/[a-z-]+` (or project-configurable endpoint prefix pattern)
- For each unique-by-spelling result, emit a list with file:line locations
- If the list has more than one unique-by-spelling but the same set of files reference both, that's a strong divergence signal

This is mechanical and would have caught the monocle case in <30s.

## Configuration

The validator should be configurable for project-specific endpoint patterns. Example `.factory/consistency-validator.config.yaml`:

```yaml
url_coherence:
enabled: true
patterns:
- "/hooks/[a-z-]+" # Phase 1 hook endpoints
- "/api/v[0-9]+/[a-z-]+" # REST API endpoints
severity: BLOCKING
canonical_source: product-brief.md # if multiple paths conflict, this artifact wins
```

## Acceptance criteria

1. `consistency-validator` agent prompt augmented to include URL-coherence check.
2. `validate-consistency` skill SKILL.md updated with the new check description.
3. Test fixture under `plugins/vsdd-factory/tests/fixtures/consistency-validator/` that reproduces the monocle UserPromptSubmit case (3 files with 2 distinct paths) and confirms the validator flags it as BLOCKING.
4. Existing project that uses the skill (`monocle`) re-runs `consistency-validation` skill and the F-NEW-01 finding (now historical) is flagged on its first commit.

## Related

- #129 — Production-grade default canonicalization (covers the broader principle that "fix in scope" requires the validator to see the defect; this issue adds one axis to the validator)
- #130 — Dispatcher recursive-shadow bug (distinct concern)

## Found during

Production-grade remediation burst on the `monocle` project (2026-05-12). The wire-protocol divergence was identified by fresh-context adversarial pass after 5 validation rounds returned clean on other axes. The fresh-context pass found it in ~30 seconds of focused reading. The fix landed via business-analyst dispatch (commit `4dfcffd` on the monocle factory-artifacts branch).

The principle: each validation axis the consistency-validator covers reduces the surface area for adversary to find new defects, but the validator has finite coverage. Adding URL-coherence is a high-leverage axis because wire contracts are load-bearing — divergence means runtime failure.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.