PolicyEngine / PolicyEngine/microcosm

SLD layer for the local artifact: per-district ACS calibration (targets, membership vintage, doctrine solve, validation)

Open
#625 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Why

State policy analysis needs district-credible breakdowns at three lenses — household, statewide, and by state legislative district. The statewide and household lenses exist today; the by-district lens does not. #449's CD experiment already taught the doctrine lesson: binding small-area targets into the national solve halved the within-10% rate. So the SLD product is a local product by construction: per-district reweighting on the local-area artifact, with the national/state calibration objective untouched (#492 embargo on new knobs applies in full — this lane changes national calibration nowhere).

Input artifact: the #512 rebuild (buildo-acs-local on the certified Build O lineage, ~1.3M households ≈ 270–670 per district across the ~4,800 SLDL+SLDU districts). The layer is built and tested against the artifact shape so it drops onto #512's release when that lane certifies; nothing here blocks or modifies #512.

Design

1. Target facts (ledger lane)

Census ACS 5-year 2020–2024 tables at SLD summary levels 610 (upper) / 620 (lower) — SLDs are published only in the 5-year dataset, so the 5-year reference window is a stated boundary of the product, not a footnote:

  • B01001 → population by age band (collapsed, sexless bands — the honest small-area grain),
  • B11001 → total households,
  • B19001 → household income brackets (the calibratable income instrument),
  • B19013 median household income → validation-only: a linear reweighting operator can honestly hit bracket counts; it cannot honestly target a median. The median is checked after the solve, never calibrated.

Witnessed source packages in the ledger (the geography levels state_legislative_district_upper/lower are already in the core vocabulary; the CD packages are the template, switched to the 5-year instrument). Verified live: all four tables serve at 610/620 with district labels carrying the "(2024)" boundary vintage; per-state pulls (national SLD wildcards are not served).

2. District membership at the ACS boundary vintage

The artifact's baked sldl/sldu columns are 2020-BAF vintage (2010-cycle districts) and donor-spine-only; the ACS 5-year tables tabulate on 2024 SLD boundaries. So the layer derives district membership itself, at the target vintage, from a new pinned source: the Census 2024 SLD BEFs (sldu24.zip/sldl24.zip, same RDO mapping-files family as the pinned cd119.zip).

One uniform seeded operator, conditioning on the finest geography each row carries:

  • donor rows (block/tract certified): direct lookup where the tract lies wholly in one district, population-weighted block draw within tract where split;
  • ACS-spine rows (no sub-PUMA geography): seeded block draw conditional on (PUMA × assigned CD × assigned county) via the block ladder, then read the block's SLDU and SLDL off the same draw — chambers stay mutually coherent and coherent with the row's existing seeded geography.

Reproducible from seed; recorded as a reviewed limitation in the same class as the artifact's existing seeded CD/county assignment for ACS rows.

3. Per-district solve (the doctrine port, #503 → US)
  • Assigned-rows-per-district: each district's candidate pool is exactly its assigned households (~270–670). The problem is block-diagonal with no shared rows or targets, so it decomposes into independent per-district solves — one uniform operator applied ~4,800 times, not 4,800 knobs.
  • Doctrine module (US analog of uk_runtime/local_doctrine.py): declared loss cap, declared stretch bound, closed scale-rule and weighting-rule vocabularies, structurally knob-free release-path signature (per-target anything is a TypeError by construction).
  • Anchor declared (#493's lesson): starting weights are the artifact's calibrated household weights restricted to the district; single-stage stretch envelope vs that anchor, realized max ratio recorded per district.
  • Past-cap census per district (#494 generalized): init/final/escaped/frozen/pushed-out per district plus a cross-district roll-up naming the worst districts.
4. Validation protocol
  • Per-district achieved-vs-target table for every calibrated cell (the release diagnostic, not a side artifact).
  • Sanity against published ACS district profiles — including the B19013 median check (achieved median within a declared within-bracket-interpolation tolerance).
  • Statewide coherence report: sum of district estimates vs the statewide artifact estimate per metric — reported, never constrained (the districts' own ACS targets already sum ≈ state totals; residual incoherence is a declared property).
  • Honest-boundaries statement generated INTO the artifact: demographics and household-income brackets are district-calibrated; tax and program detail is inherited from the state-calibrated solve; ACS 5-year 2020–2024 reference window; seeded-assignment class for ACS-spine rows; small-area tail caveats on the register columns.
Product shape

A sidecar in the release directory (per-chamber long weights (area_type, area_code, household_id, weight) + per-district diagnostics + doctrine/census JSON), registered as non-microdata artifacts in the release manifest with sha256sums coverage — riding the #398 non_default_local_area contract, which is require-based and validates sidecars that declare themselves.

Pilot: Utah, then generalization

Pilot on Utah: both chambers present (unlike NE), single-member districts throughout (no NH floterial or multi-member representation quirks), maps adopted 2021 and stable through the 2024 tabulation vintage (clean external-profile comparison), 29 SLDU + 75 SLDL = right pilot scale, and an active state-policy engagement to serve. Generalization is the same code over the state enumeration, with the special cases declared per state (NE unicameral, DC wards-as-SLDU, NH floterial tabulation, multi-member districts as Census tabulates them).

Deliverables

  1. Ledger: SLD source packages (pilot scope, then national) — witnessed, pinned, 5-year vintage declared.
  2. Populace: SLD membership operator (2024-BEF vintage) + per-district solver under doctrine + past-cap census; TDD against the artifact shape; no national-calibration diffs.
  3. Validation protocol + honest-boundaries statement wired into the sidecar outputs.
  4. PRs with cross-family review; merge is the owner session's call.

Sequenced behind #512 (its lane owns the rebuild; this lane consumes the artifact). Relates #449, #492, #493, #494, #398, #503.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the #512 buildo-acs-local artifact contract, the UK doctrine reference in uk_runtime/local_doctrine.py, and the #398 non_default_local_area sidecar contract. Implement the Utah pilot's ledger, 2024-BEF membership operator, per-district doctrine solve, diagnostics, and validation as a TDD-tested layer without changing national calibration. Done means sidecar weights, diagnostics, doctrine/census JSON, honest-boundaries output, manifest registration, and sha256 coverage are present.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.