PolicyEngine / PolicyEngine/microcosm
SLD layer for the local artifact: per-district ACS calibration (targets, membership vintage, doctrine solve, validation)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 4
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 94
Description
Why
State policy analysis needs district-credible breakdowns at three lenses — household, statewide, and by state legislative district. The statewide and household lenses exist today; the by-district lens does not. #449's CD experiment already taught the doctrine lesson: binding small-area targets into the national solve halved the within-10% rate. So the SLD product is a local product by construction: per-district reweighting on the local-area artifact, with the national/state calibration objective untouched (#492 embargo on new knobs applies in full — this lane changes national calibration nowhere).
Input artifact: the #512 rebuild (buildo-acs-local on the certified Build O lineage, ~1.3M households ≈ 270–670 per district across the ~4,800 SLDL+SLDU districts). The layer is built and tested against the artifact shape so it drops onto #512's release when that lane certifies; nothing here blocks or modifies #512.
Design
1. Target facts (ledger lane)
Census ACS 5-year 2020–2024 tables at SLD summary levels 610 (upper) / 620 (lower) — SLDs are published only in the 5-year dataset, so the 5-year reference window is a stated boundary of the product, not a footnote:
- B01001 → population by age band (collapsed, sexless bands — the honest small-area grain),
- B11001 → total households,
- B19001 → household income brackets (the calibratable income instrument),
- B19013 median household income → validation-only: a linear reweighting operator can honestly hit bracket counts; it cannot honestly target a median. The median is checked after the solve, never calibrated.
Witnessed source packages in the ledger (the geography levels state_legislative_district_upper/lower are already in the core vocabulary; the CD packages are the template, switched to the 5-year instrument). Verified live: all four tables serve at 610/620 with district labels carrying the "(2024)" boundary vintage; per-state pulls (national SLD wildcards are not served).
2. District membership at the ACS boundary vintage
The artifact's baked sldl/sldu columns are 2020-BAF vintage (2010-cycle districts) and donor-spine-only; the ACS 5-year tables tabulate on 2024 SLD boundaries. So the layer derives district membership itself, at the target vintage, from a new pinned source: the Census 2024 SLD BEFs (sldu24.zip/sldl24.zip, same RDO mapping-files family as the pinned cd119.zip).
One uniform seeded operator, conditioning on the finest geography each row carries:
- donor rows (block/tract certified): direct lookup where the tract lies wholly in one district, population-weighted block draw within tract where split;
- ACS-spine rows (no sub-PUMA geography): seeded block draw conditional on (PUMA × assigned CD × assigned county) via the block ladder, then read the block's SLDU and SLDL off the same draw — chambers stay mutually coherent and coherent with the row's existing seeded geography.
Reproducible from seed; recorded as a reviewed limitation in the same class as the artifact's existing seeded CD/county assignment for ACS rows.
3. Per-district solve (the doctrine port, #503 → US)
- Assigned-rows-per-district: each district's candidate pool is exactly its assigned households (~270–670). The problem is block-diagonal with no shared rows or targets, so it decomposes into independent per-district solves — one uniform operator applied ~4,800 times, not 4,800 knobs.
- Doctrine module (US analog of
uk_runtime/local_doctrine.py): declared loss cap, declared stretch bound, closed scale-rule and weighting-rule vocabularies, structurally knob-free release-path signature (per-target anything is aTypeErrorby construction). - Anchor declared (#493's lesson): starting weights are the artifact's calibrated household weights restricted to the district; single-stage stretch envelope vs that anchor, realized max ratio recorded per district.
- Past-cap census per district (#494 generalized): init/final/escaped/frozen/pushed-out per district plus a cross-district roll-up naming the worst districts.
4. Validation protocol
- Per-district achieved-vs-target table for every calibrated cell (the release diagnostic, not a side artifact).
- Sanity against published ACS district profiles — including the B19013 median check (achieved median within a declared within-bracket-interpolation tolerance).
- Statewide coherence report: sum of district estimates vs the statewide artifact estimate per metric — reported, never constrained (the districts' own ACS targets already sum ≈ state totals; residual incoherence is a declared property).
- Honest-boundaries statement generated INTO the artifact: demographics and household-income brackets are district-calibrated; tax and program detail is inherited from the state-calibrated solve; ACS 5-year 2020–2024 reference window; seeded-assignment class for ACS-spine rows; small-area tail caveats on the register columns.
Product shape
A sidecar in the release directory (per-chamber long weights (area_type, area_code, household_id, weight) + per-district diagnostics + doctrine/census JSON), registered as non-microdata artifacts in the release manifest with sha256sums coverage — riding the #398 non_default_local_area contract, which is require-based and validates sidecars that declare themselves.
Pilot: Utah, then generalization
Pilot on Utah: both chambers present (unlike NE), single-member districts throughout (no NH floterial or multi-member representation quirks), maps adopted 2021 and stable through the 2024 tabulation vintage (clean external-profile comparison), 29 SLDU + 75 SLDL = right pilot scale, and an active state-policy engagement to serve. Generalization is the same code over the state enumeration, with the special cases declared per state (NE unicameral, DC wards-as-SLDU, NH floterial tabulation, multi-member districts as Census tabulates them).
Deliverables
- Ledger: SLD source packages (pilot scope, then national) — witnessed, pinned, 5-year vintage declared.
- Populace: SLD membership operator (2024-BEF vintage) + per-district solver under doctrine + past-cap census; TDD against the artifact shape; no national-calibration diffs.
- Validation protocol + honest-boundaries statement wired into the sidecar outputs.
- PRs with cross-family review; merge is the owner session's call.
Sequenced behind #512 (its lane owns the rebuild; this lane consumes the artifact). Relates #449, #492, #493, #494, #398, #503.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the #512 buildo-acs-local artifact contract, the UK doctrine reference in uk_runtime/local_doctrine.py, and the #398 non_default_local_area sidecar contract. Implement the Utah pilot's ledger, 2024-BEF membership operator, per-district doctrine solve, diagnostics, and validation as a TDD-tested layer without changing national calibration. Done means sidecar weights, diagnostics, doctrine/census JSON, honest-boundaries output, manifest registration, and sha256 coverage are present.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100