PolicyEngine / PolicyEngine/microcosm
UK graph: move geography assignment ahead of enrichment (geography-first reordering)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 4
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 94
Description
Why. The release path (docs/us-uk-release-path.md) fixes one construction order for both countries: complete multispine including its initial clones → assign one atomic area per household → derive larger geographies from versioned mappings → enrichment that retains the assigned location → calibration → full and compact exports. The UK full-build graph registered by #901 keeps the older UK order: every spine stage runs on the source FRS before cloning, the K geographic clones are created after the spine, and the legacy ladder draw assigns geography just before target compilation. That draw is not keyed to household identity (row order or K changes every draw), the ladder carries no machine-readable exact / best-fit relation per layer, and its publisher provenance is only partly committed. This issue tracks the PRs that move the UK to the shared order and the shared operators, each with its own before/after evidence.
Fixed decisions (2026-09-14/15). Atomic level stays the lowest published census unit per nation: E&W 2021 Output Areas (188,880), Scotland 2022 Output Areas (46,363), NI 2021 Data Zones (3,780). Sampling law becomes a single-stage draw of the atomic area by census household count within the household's FRS region. Vintages for FY2024-25: LAD April 2023 code set (the 1 April 2025 Barnsley/Sheffield recode is item (e) below), ITL 2025 for all nations, Ward 2024, Westminster constituencies July 2024. K, seed and solve doctrine are unchanged by any PR in this list. The UK geography.yaml stays phase: legacy until item (f).
Tracked PRs (checklist).
- (a) Assignment swap (step 2; stacked on #901; draft PR #932): replace
uk.full.locations/uk.full.geography_mappingwith the sharedatomic_geography_nodes(support import, assign, derive, gate) keyed bygeography_household_keyfrom the existinghousehold_draw_keylineage; single-stage law in the UK adapter; three support artifacts rebuilt from published lookups with a committed provenance JSON andsources.yamlentries;--geography-assignment legacykept for measurement builds only (--release-candidaterefuses it); assignment-level before/after on {legacy two-stage, keyed single-stage} × {K=1, K=15} with per-area evidence (887 format). No calibration A/B. - (b) Enrichment reorder, one area-dependent stage at a time (frs_brma, was_wealth, lcfs_consumption, regional_uprating, council tax), each with a distribution receipt against the pre-move build; decide per stage whether it conditions on assigned local authority, constituency or region. Weight-changing stages (SPI support channel, CGT clones) must stay before the origin-budget freeze point.
- (c) Origin budget and caps: decide the budget's reference weight once (b) settles how weights flow into calibration; decide whether the UK calibrated node adopts the US two-cap declaration (solver bound against incoming importance weights plus the executor's design cap). No solve-doctrine change lands with (a) or (b).
- (d) Rosters and area support from the supports: derive the 650/361 roster and the terminal area-support summary from the three support artifacts; retire the ladder NPZ and
local_area_crosswalk.json's ladder pins. - (e) LAD 2025 recode: vintage translation for Barnsley E08000016→E08000038 and Sheffield E08000019→E08000039 in the #205 pattern, when 2025-coded facts arrive in Chronicle.
- (f) F3
geography.yamlbundle diff: move the UK spec fromphase: legacytophase: f3declaring the OA anchor, single-stage draw, shared kernels, per-layer vintages, typed source refs for the three supports and a keyed seed stream; needs two shared spec-engine extensions (a support list field ingeography.schema.json, a keyed seed site inseeds.py), reviewed with Max. - Chronicle: issue PolicyEngine/chronicle#269 and its raw-only registration PR for the sixteen publisher lookup files (separate lane).
Out of scope for this issue. Calibration doctrine, K, the UC element and Chronicle schema work (#882, chronicle#259/#261/#262), and the US pipeline.
Evidence. Each PR records its receipts under experiments/931-… and per-cell evidence under docs/evidence/uk-931/, with sha256s in the PR body. Nothing here certifies a release; the release battery and holdout stay separate gates.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Begin with draft PR #932 and the existing UK full-build graph from #901, then inspect the shared atomic_geography_nodes operators and the geography_household_key lineage. Read docs/us-uk-release-path.md, geography.yaml, geography.schema.json, and seeds.py, and review the experiments/931-* and docs/evidence/uk-931/ paths. Done requires the tracked stages, support artifacts, provenance, evidence receipts, and F3 specification changes to be completed without changing the stated solve doctrine.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100