Gate 2 governance: blast-radius scoring, test mutation validation, and multi-reviewer consensus on critical surfaces
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 28/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- typescript
- Domain
- ci-cd, developer-experience, security, testing
Research direction
Start with docs/concepts/method.md for the existing Gate 2 model, then inspect packages/harness/src/blast-radius.ts and the named critical-surface paths. Define how blast-radius scoring, mutation validation, and dual-reviewer approval connect to delivery state transitions. Done means high-risk changes are escalated, weak tests are flagged, and critical surfaces require the proposed human consensus.
Written by the indexing model from the issue text.
Description
Problem
Facility's operating model places absolute trust in Two Inviolable Human Gates (docs/concepts/method.md):
- Gate 1: Human judges and accepts the
/architectplan. - Gate 2: Human validates the preview and performs the squash-merge. "Every merge carries a human decision."
While Gate 2 successfully prevents runaway autonomous pushes, scaling agent delivery reveals a fundamental governance gap: Gate 2 treats all PRs as equal-risk and assumes green CI checks equal meaningful test assertions.
What breaks at scale
- Single-reviewer alert fatigue: When an agent swarm delivers 10+ PRs daily, a single reviewer inevitably skims diffs and leans on green CI checks as proof of correctness.
- Tautological agent tests: LLMs frequently write test suites that achieve 100% line coverage while asserting weak or self-fulfilling invariants. A flawed implementation accompanied by a flawed test suite passes all CI checks cleanly.
- Flat risk policy: A documentation typo fix and a PR modifying
services/gateway/src/auth.ts,guards/, orpackages/db/src/schema.tspass through the exact same single-maintainer squash-merge flow.
If a single reviewer is fatigued, rushed, or compromised, green CI status allows subtle regressions or security backdoors on critical surfaces to merge undetected.
Proposed architecture
We propose hardening Gate 2 with three tiered governance mechanisms:
[Agent PR Created]
│
▼
┌───────────────────────────────────┐
│ 1. Diff Blast-Radius Scorer │
└───────────────────────────────────┘
├── Low Risk (docs / UI) ─────► Standard Gate 2 (Single Human)
└── High Risk (auth / db / guards) ──┐
▼
┌───────────────────────────────────┐
│ 2. Adversarial Mutation Runner │
└───────────────────────────────────┘
│
▼
┌───────────────────────────────────┐
│ 3. Dual-Reviewer Consensus Gate │
└───────────────────────────────────┘
1. Diff Blast-Radius Scoring (packages/harness/src/blast-radius.ts)
Introduce deterministic risk scoring for every delivery branch based on modified paths, entropy, and sensitive surfaces:
- Critical surfaces:
guards/,.github/workflows/,packages/db/src/schema.ts,services/gateway/src/auth.ts,services/api/src/idempotency.ts,STANDARD.md. - Medium surfaces: Backend service endpoints, state machines, SDK contracts.
- Low surfaces: UI styles, documentation, static assets.
- If a delivery touches a Critical Surface, the platform flags the story as
high_riskand escalates review requirements.
2. Automated Adversarial Mutation Validation
To ensure agent-generated tests actually assert business invariants rather than green execution paths:
- When a PR adds or modifies tests, run a lightweight AST mutation step in the sandbox (e.g. inverting conditionals, flipping boolean returns, stripping guard clauses).
- The Ratchet Rule: If the agent's new test suite still passes against intentional mutations, the run fails with
suspicious_weak_testsand requires the agent to add assertions that catch the mutants before opening/updating the PR.
3. Multi-Reviewer Consensus on Critical Surfaces
- Escalated
high_riskstories require approvals from two distinct human reviewers (or a designated security owner) before the story can transition toDoneor merge intomain. - Prohibit single-maintainer squash-merges on changes modifying cryptographic operations, virtual key routing, or guard policies.
Expected impact
- Zero silent regressions on core infrastructure: Critical surfaces gain defense-in-depth protection against single-point-of-failure human reviews.
- Auditable test integrity: Verifies that agent tests catch real regressions rather than providing false confidence.
- Sustainable reviewer throughput: Low-risk changes continue shipping fast with single-human approval, focusing human attention strictly where the blast radius is large.
- Dominant language
- TypeScript
- Stars
- 71
- Forks
- 64
- Avg merge
- 15h 38m
- Merged PRs (30d)
- 66
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from theam/facility
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
enhancement
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Similar issues
-
clawsweeper:fix-shape-clear clawsweeper:queueable-fix clawsweeper:source-repro impact:ux-friction issue-rating: 🦞 diamond lobster no-stale P3
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
community first-timers-only good first issue hacktoberfest help wanted low hanging fruit up-for-grabs
Difficulty 1/5 Under an hour Newbie friendliness 76/100
-
code-quality refactoring
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
github/gh-aw-firewall#8816 ·
-
integration:quickjs org:external priority:backlog topic:code-interpreter topic:middleware type:feature
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
langchain-ai/deepagents#6450 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
vercel/react-tweet#225 ·