proposal: distribute tests across rungs by product quality and severity, not cost alone

Context

The six-rung ladder (#915) assigns tests to rungs by cost and hit rate (#576): a gate earns a local slot if it fails more often than local_cost / 738s. That is one element. Now that the rungs are owned by governance, the other element matters more: which product qualities are proven how early, so that the critical ones are falsified before a 48-hour agent run builds a hundred commits on top of a wrong assumption.

The repo already has the vocabulary. tests/claims.json#vocabulary.qualities names eleven qualities (correctness, contracts, durability, concurrency, performance, scalability, security, offline, compat, journey, accessibility) and a severity scale (S1 sovereignty or data at stake, S2 silent wrongness, S3 a promise degraded, S4 fidelity). The ladder never used either. Gates are classed product / contract / hygiene in scripts/ci/gate-classes.json, which says what kind of claim a gate makes, not which quality it protects or how bad a miss is. Two taxonomies, unconnected.

#988's branch tier moved contract checks and product tests off non-main pushes. Right for a human who sees CI in twelve minutes; wrong for an agent that builds on a branch for two days.

Split from #1005, which owns the directive primitive, the door field, and the front page this proposal's matrix renders on.

Decision

Each product quality is proven at the lowest rung where a test can falsify it on the diff. When a rung's budget is contended, severity decides who keeps the slot. That is the TESTING.md axiom's "cheapest tier that can falsify it" applied per quality instead of per test; the #576 economics remain the tiebreaker within a rung, never the reason to move a quality between rungs.

Floors, and what sets them:

Quality	Lowest feasible rung	What sets the floor
Contracts	0	Staged-file lint decides it; the rest of the contract bundle joins at 1 on every branch
Security, static half	0–1	Vault SQL reach, secret scanning, consent-ledger shape are text checks on the diff
Correctness	1, every branch	Affected unit and contract tests; today only main pushes pay this
Security, dynamic half	2	Isolation and consent laws need the built gateway
Durability, concurrency	2	Need a real vault and a real log; scoped runs fit rung 1 for the package touched
Performance	2 for budgets, 3 for the strace gate	Needs a measured run on the reference container
Journey, accessibility	3–4	Need a built app on a device or browser
Scalability, offline	4	Scale and chaos rigs need time and a candidate
Compat	5	Needs live dependencies

Every gate and every test carries its quality, and the quality-by-rung matrix is computed from that metadata and shown on the front page — never typed by hand. A diff that touches an S1 or S2 quality with no falsifier at rung 1 is a finding. This is the confidence map in TESTING.md, computed instead of asserted.

Correctness and contracts at rung 1 on every branch reverses the #988 branch tier for product tests. Gate stamps (#988) already make this cheap: an agent pays only for trees it has not yet proven.

Size

Single PR — one agent, one focused diff

Scope

In:

• quality per gate in scripts/ci/gate-classes.json and per test file via the existing claims machinery; gate-classes.test.mjs fails on an unclassified quality.
• The quality-by-rung matrix generator and its front-page rendering (rendered by #1005's formatter; until that lands, a check summary).
• The S1/S2-with-no-rung-1-falsifier finding.
• check:push on non-main branches restored to lint:product plus affected product tests; docs/dev-environment.md push-tier table updated.
• Rung moves implied by the table, each recorded in tests/budgets.json and gate-classes.json with the quality that justifies it.

Out:

• Changing what any test asserts, or weakening any ratchet or budget.
• New tests. This assigns and exposes; gaps it reveals become their own issues.
• The directive primitive, doors, and the front page itself — #1005.

Acceptance criteria

• Every gate in gate-classes.json carries a quality from the claims vocabulary; the test fails on an unclassified gate.
• The quality-by-rung matrix is generated from metadata, matches the table above for every quality, and is visible on the PR (front page or check summary).
• A fixture diff touching an S1 quality with no rung-1 falsifier produces a finding; a fixture diff with one does not.
• Non-main pushes run lint:product and affected product tests; a second push of an unchanged tree is skipped by the stamp; measured on #1002's range and recorded in the receipt.
• docs/dev-environment.md rung table and push-tier section, and TESTING.md's confidence map, describe the computed state and point at the generator.
• No test assertion, budget, or floor weakened; bun run lint:ledgers green.

Validation

• gate-classes.test.mjs and the matrix generator's tests at rung 0.
• Fixture diffs for the S1 finding, both directions.
• Push-tier wall clock on #1002's range with and without stamps, in the receipt.
• bash .governance/run.sh and bun run check:pr green.

Open questions

None. Ruled under #1005's brainstorm: earliest-rung-where-falsifiable, severity as tiebreaker, cost as tiebreaker within a rung only.

Start with scripts/ci/gate-classes.json, gate-classes.test.mjs, tests/claims.json, tests/budgets.json, and the existing claims machinery; read TESTING.md and docs/dev-environment.md for current rung rules. Run the named gate and matrix-generator tests at rung 0, then validate fixture findings and check:pr/check:push behavior. Done means the generated matrix, findings, branch checks, docs, and ledger validations meet the acceptance criteria without changing assertions or budgets.