PolicyEngine / PolicyEngine/microcosm

Close the schema-3 terminal-report path for non-grandfathered exact-k UK releases

Open
#712 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Found by the #706 adversarial review (verdict: needs-attention; dispositioned in the PR's review-response comment). The data-shard release verifier still accepts schema-3 legacy terminal reports for any exact-k UK release, which lets a release bypass the spec-armed weighted-integrity gates #706 lands.

Mechanics

validate_release_dir dispatches on the terminal report's own schema_version (contract.py, the vintage-dispatch block): 4 → _check_uk_gate_battery_report (full battery: pinned digests, pinned thresholds per #706, entry-membership checks), 3 → _check_uk_terminal_gate_report (the pre-#691 legacy aggregator checker). Nothing restricts schema 3 to grandfathered releases. The legacy checker derives gate membership from the build manifest's terminal_gate_evidence listing, so a new exact-k release shipping a signed schema-3 report that omits the input-mass and QRF evidence stages is verified against a gate set that never includes them — the gates #706 made release-blocking are skipped entirely.

Severity honestly stated

Both checkers require a valid HMAC-SHA256 signature under the out-of-band UK terminal-gate signing key, which is never committed and absent from secrets-free PR CI. No keyless actor can fabricate an acceptable report in either schema, so this is not an open door: the reachable failure mode is a trusted-but-mistaken publish flow — a stale checkout or old tooling producing an honestly-signed schema-3 report and slipping a release past gates it should face. Defense-in-depth gap: the verifier should enforce policy against operator error, not just forgery. Pre-existing on main (predates #706); #691 retired the schema-3 path on the producer side only, and the data-shard verifier deliberately kept it "defensively" for grandfathered reports.

Fix

In the vintage dispatch: accept schema_version == 3 only when release_id in _UK_LEGACY_RELEASE_IDS (today: the June release only — which is not exact-k and never reaches this branch, so in practice schema 3 is refused on the exact-k path outright); otherwise fail with a message requiring the schema-4 gate battery report. Add a regression test that a new exact-k release with a signed schema-3 report is refused.

The real cost: the schema-3 fixture suite (35 tests)

test_contract.py's standard fixture (_write_uk_release_dir) fabricates an exact-k release whose terminal report is schema-3, and 35 tests drive validate_release_dir through it (forgery/wrong-key/transplant/rehash rejections, tier checks, diagnostics shapes, weight-ratio observables). All 35 fail at the new refusal before reaching their behavior under test. Each needs a case-by-case disposition:

  1. Migrate to the schema-4 battery fixture (_write_battery_release) — most forgery/signature scenarios; first dedup against existing schema-4 twins.
  2. Re-anchor the schema-3-specific mechanics tests (legacy policy digests, schema-3 attestation) to direct _check_uk_terminal_gate_report calls or the grandfathered June fixture — the legacy checker stays alive for the grandfathered id.
  3. Retire the few whose scenario is subsumed by the new refusal test.

That triage is review-sensitive, half-day-scale work — the reason this was deferred out of #706 rather than ridden along. It is the data-shard completion of the #654 schema-3 retirement.

Refs #706 (adversarial-review disposition), #654/#691 (producer-side retirement), #630.

🤖 Generated with Claude Code

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in contract.py at the vintage-dispatch block in validate_release_dir, then inspect test_contract.py and the _write_uk_release_dir and _write_battery_release fixtures. Confirm that a signed schema-3 report for a new exact-k release is refused, then migrate, re-anchor, or retire the 35 affected tests as appropriate and run the contract test suite.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
release, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
54/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.