PolicyEngine / PolicyEngine/microcosm
Rebuild the US ACS local-area artifact on the certified Build O lineage (buildl 36de5d9a is stale)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 4
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 94
Description
Why
The only local-area product in circulation is populace-us-2024-buildl-acs-local-36de5d9a-20260712T104640Z (built 2026-07-12 from branch buildl-acs-spine @ 32d446b, donor = the June dense base). It predates every target-surface repair of the Build M/N/O campaign:
- capital-gain-distributions defect era (the +635% signature; pre-memo-leg, pre-truthful CG/interest chains),
- pre-#469/#488 truthful-surface work and the #491 shared critical-target register,
- old SSI machinery (pre one-shot Bernoulli seeding),
- no adult-care / SE-health inputs (#451 items 1–2, landed in #500),
- a donor that cannot satisfy the 163-column input-coverage surface base-O now proves live.
policyengine.py's dataset_overlays.us.populace_us_2024_acs_local still pins this stale artifact (sha 36de5d9a…). The QRF transfer replicates the donor distribution onto the ACS spine, so every donor-side repair since July 12 is absent from the only local-area surface consumers can name.
Plan
Rebuild the ACS local artifact on the certified Build O lineage — base-O (base_populace_us_2024_puf_support.h5, sha 356cf501e1ca…, 23 stages, block/tract/CD geography assigned) — the same lineage as the certified default populace-us-2024-buildo-sparse-rmloss100-22bd902-20260722T232627Z.
- Port the Build L tooling into
tools/as first-class code (it lives on the unmergedbuildl-acs-spinebranch +_buildl-runtimescripts today):acs_pums/acs_sources/acs_inputs/acs_transfer/acs_multispine/base_poolmodules with their test suites, the multispine staging driver, and the materialize→calibrate→finalize→package chain — so the next refresh is one command. - Donor semantics updated for the 23-stage base: the QRF plan's PUF-detail families auto-extend from the current constants; explicit families extend to the new donor-observed inputs (adult-care, SE-health); buildl-era exclusions (
partnership_self_employment_net_earnings,s_corp_income) re-verified against base-O. - Geography: donor rows keep base-O's certified block-ladder assignment (block/tract/CD/county/place/SLD/CBSA) instead of buildl's state-conditional PUMA re-draw; donor
pumaderives deterministically fromtract_geoidvia the PUMA ladder's tract-overlap table (2020 tracts nest in exactly one 2020 PUMA). ACS rows keep buildl semantics: observed PUMA → seeded CD/county assignment. - Calibration surface unchanged (no new knobs, #492 embargo): state+CD population marginals + state SNAP benefits/caseloads + state Medicaid enrollment (RI substitution) + state SOI, compiled from the current feed vintage (v9.3,
eab45a2b…); 800 epochs / 400-epoch warm-start batches / ratio 5.0 / lambda 0 / seed 0 / mass conserve — the Build L runbook decisions verbatim. - Publish per the local product's contract: implement #398's dataset-role classes (
non_default_local_areagets its own validation: gates object + pass, diagnostics shape, source coverage keyed to the families actually calibrated, reviewed-limitations register; pointer ops forbidden), then publish through the tracked path — immutable tag onpolicyengine/populace-us,latest.jsonuntouched. - policyengine.py overlay repin PR: new revision + sha256 in
dataset_overlays.us.populace_us_2024_acs_local; the overlay-preserved / default-resolution-unchanged semantics tests stay intact.
Known boundaries the release will declare
- #507 SSI aged-band collapse, inherited by construction — base-O carries the 65+ take-up banding defect (aged baseline 0.94M vs SSA 2.42M). This rebuild does not wait for the fix: the committed one-command chain makes the O-2-based refresh mechanical once #508's fix certifies.
- #403's buildl QA findings re-measured, not assumed fixed: per-spine SSI incidence, weight-composition skew, and ESS get measured on the new artifact and recorded in its diagnostics.
- #393 miscellaneous_income loss-side donor defect (open): restated as inherited via transfer if still measured on this lineage.
- CD population marginal vintage 2020 (honest v1); ACS-spine GQ-housing structural nulls; mixed-coverage sub-PUMA columns (donor rows carry block/tract, ACS rows do not); ESS concentration under the hard 5× cap with lambda 0.
Receipts
Build receipts land here and on #449 (the Build O battery thread). Relates #48, #398, #403, #449, #492, #507.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the unmerged buildl-acs-spine branch, its _buildl-runtime scripts, and the tools/ modules named in the plan. Trace the multispine staging driver and materialize→calibrate→finalize→package chain, then inspect policyengine.py's dataset_overlays.us.populace_us_2024_acs_local. Done means a validated Build O artifact, published tracked metadata, and an overlay repin with the stated semantics tests intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, data-engineering, release, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100