PolicyEngine / PolicyEngine/policyengine-scorecard

policyengine-uk reform-validation population: per-release regression history for the UK engine

Open
#62 0 comments 0 reactions 1 assignee View on GitHub

@vahid-ahmadi is already working on this.

Since Aug 17, 2026.

Dominant language
Python
Stars
0
Forks
0
Avg merge
6d 12h
Merged PRs (30d)
32

Description

The third US population — the populace reform-validation registry (#15/#16, scorecard_db/ingest_reform_validation.py, surfaced in the app by #22) — gives every certified populace-us release a per-release regression history against external checks. The ingest is hardcoded to populace-us releases (its RELEASES map: 1.729.0 → 1.764.6). policyengine-uk has no equivalent: pe-uk 2.89.x releases and new certified populace-uk bundles land with no Scorecard record of whether previously-matching UK rows still match.

The UK side is about to have exactly the row inventory this needs: mode-1 counterparts (#40/#51), OBR costings (#56), the reckoner (#60), UC deductions (#21/#39/#52). Without a per-release population, every one of those comparisons is a snapshot that silently rots as the engine moves.

Design — same contract as the US population, parameterised by country

  • A reform_validation.json-shaped artifact per (pe-uk release × certified populace-uk bundle), produced by re-running the registered UK reform/counterpart set offline on the managed environment (the #51/#56 pipelines already emit per-run artifacts with engine_version + bundle id — the release history is largely a re-keying of what they save).
  • Extend ingest_reform_validation.py to carry country and a UK release map rather than forking a UK sibling — the schema layer #32 adds (top-level country) is the natural base, another reason #57 resolves first.
  • App: the reform-validation tab's per-release history view gets the country dimension via #42/#50.
  • Selection rule for the battery: every UK row whose status is ok/match_* at time of registration; regressions (a row leaving tolerance on a new release) are first-class events, not silent re-snapshots.

Why now

Fiscal events force engine churn (Autumn Budget 2026 parameter updates land in pe-uk within weeks of the event), and the two-child-limit abolition is exactly the kind of change that flips case expectations (see the #49 case pair). This population is what makes the UK Scorecard continuous (#7) rather than a one-time harvest.

Depends on: #57 (country-carrying ingest), #51 (artifact provenance), first UK managed runs. US precedent: #16, #22.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.