PolicyEngine / PolicyEngine/microcosm
UK output-area-anchored geography ladder (US block-ladder pattern)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 4
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 94
Description
UK output-area-anchored geography ladder (the US block-ladder pattern for the UK)
Goal
Give the UK artifact the same local-geography spine the US artifact has (us_runtime/geography_ladder.py): anchor every household at the finest published census geography, derive every other layer deterministically from that anchor, and keep one national dataset, filterable at any grain (the #275 standing rule — no per-area files).
Design (mirroring the US block ladder)
Anchor = census output area (OA). The UK's block equivalent: 2021 Census OAs for England & Wales, Scotland's Census 2022 OAs, NI small areas. Assignment ladder:
- Region is already assigned (FRS carries it; it is a calibrated dimension of the national build).
- Assign a Westminster parliamentary constituency (2024 boundaries) within region, sampled proportional to constituency household counts — the politically salient rung, and the rung where per-area calibrated weights (#146 long weights) can later bind, exactly as the US anchors its ladder inside the assigned congressional district.
- Assign one OA within the constituency, sampled proportional to OA population (ONS OA→PCON24 best-fit lookup defines membership).
- Every other layer derives from the OA — no independent randomness:
- LSOA and MSOA: structural nesting (OA → LSOA → MSOA).
- Local authority district: ONS OA→LAD lookup.
- Ward: OA→ward lookup.
- ITL levels / combined authorities: LAD-level lookups.
UK geographies famously do not nest cleanly (constituencies cross LAD boundaries); the ONS Open Geography Portal best-fit lookups are the sanctioned resolution and should be carried in the ladder artifact itself, sha-pinned, like the US ladder artifact carries its block crosswalks.
Discipline ported from the US implementation
- Vintage policy = error: the ladder artifact records one vintage per derived layer; the loader refuses an artifact missing any; assignment refuses a ladder whose constituency vintage differs from the vintage the constituencies were assigned under (the #205 lesson — no silent partial joins on mismatched vintages).
- Engine inputs only: exported columns are policyengine-uk household inputs (or plain data columns where no engine input exists); never persisted formula outputs (the #34 lesson — derived quantities recompute from the rung).
- Seeded draws: constituency and OA sampling under the build's seed discipline, reproducible from the release manifest.
Relationship to the existing UK local machinery
Complementary, not competing, with #146/#147:
- The ladder gives the spine: every household locatable at any grain, local analysis = filter (or aggregate) the national file.
- The long weights (#146,
local_solver/local_runner, already ported) give calibrated per-area estimate quality on top of the spine where area targets exist. - The credibility gates (#147) score whether a given area's filtered/weighted estimate is publishable — random-within-region assignment is only as good as region-level joint structure, and the gates are what say so per area.
Acceptance
- Ladder artifact (OA anchor + lookups + vintages) built from ONS sources, sha-pinned, with provenance in the ledger.
- Assignment wired into the UK build behind the country-spec geography-spine schema; gate output mirrors
us_geography_ladder_assignment_summary. - National artifact round-trips: filter by constituency/LAD/region reproduces the calibrated region totals; assignment summary reports per-rung coverage.
- Cross-checks: constituency household counts vs ONS, LAD population shares vs mid-year estimates (report-only at first; gates per #147).
First milestone: England & Wales ladder end-to-end on the current populace-uk 2023 artifact; Scotland/NI rungs follow once their lookup vintages are pinned.
Refs: #146 (local variant publication), #147 (credibility gates), #145 (source refresh), #275 (one-national-dataset rule), populace#8 / policyengine.py#427 (the artifact this extends).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with packages/populace-build/src/populace/build/us_runtime/geography_ladder.py, then trace the UK build's country-spec geography-spine schema and the existing local_solver/local_runner machinery. The first milestone is an England and Wales ladder for the current populace-uk 2023 artifact; done means the sha-pinned artifact, assignment summary, national round-trip, and stated cross-checks pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100