PolicyEngine / PolicyEngine/microcosm
County-calibrated County Files (3,144 counties, 2026 + 2027): build epic
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 4
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 94
Description
County-calibrated US files for all 3,144 counties, years 2026 and 2027, delivered as one HDF5 per year with the six entity tables. Scope committed externally; MVP target is 6–10 weeks from the agreement's effective date (mid-September 2026). This epic collects what the repo has and what it lacks, verified on 2026-09-03 against origin/main and the installed engine.
What exists
- Income-blind county assignment on both geography lanes:
us_runtime/puma_ladder.py(county sampled within PUMA by block-population overlap) andus_runtime/geography_ladder.py(block draw within CD;county_fips = block_geoid[:5]). Both writecounty_fips,sldu,sldl,cbsa_code, etc. onto the household table. - Occupation already on every person record (
detailed_occupation_recode,org_wagesstage). - Engine inputs for industry recodes and worked-last-year merged in policyengine-us #9311; the pin bump landed in #756.
- Writers are period-parametric (
write_dataset(..., period=),write_nullable_us_h5(..., period=)).
What is missing
- County target surface and loss rung.
packages/microcosm-calibrate/has no county target family, no county fact ingestion, no county rung; the ACS-local tool calibrates state + CD only (tools/build_us_acs_local_release.pycolumn_map). Needed: ACS 5-year county income tables (B19001 / B19013 / S2001), IRS SOI county, Census county population; Ledger ingestion; multi-level consistency national → state → county. - Support. Pooled base ≈ 338k households over three ASEC years, ~107 per county on average before L0 pruning. Decide pool scale and pruning policy for county grain.
- 2023 county geography. The block ladder is built on 2020 tabulation blocks (
block_vintage: 2020_tabulation_blocks), so Connecticut resolves to the eight legacy counties (3,143 areas). The commitment is 2023 geography with the nine planning regions (3,144). Needs a 2020-block → 2023-county crosswalk artifact and a re-cut ladder. - Non-2024 periods.
PERIOD = 2024is hard-coded intools/build_us_fiscal_refresh_release.pyandtools/build_us_acs_local_release.py; no US release has been produced at another period. 2026 needs aged targets (us_runtime/target_aging.pyexists; confirm production wiring) and 2027 is that again on projected targets. - Industry and worked-last-year on the records. #745 (
work_experience_inputsstage; parent #719) is still a draft with no CI; it needs a rebase onto post-#756 main, bundle/ABI regeneration, and a regenerated ASEC pooled base since the frozencensus_cpsinputs lack WEIND/WEMIND. - Household measures as materialized values.
household_net_incomeandhousehold_market_incomeare formula-owned (adds/subtracts), and the export gate refuses formula-owned columns in entity tables (adapters/policyengine_us.py, "Export contract violated"). The delivery commitment is earnings, gross income, and net income as materialized values keyed byhousehold_id, readable without running the engine: ship a derived-measures table alongside the entity tables (release-directory sidecar or a non-input HDF5 key) computed at the delivery period. weeks_workedat 2026/2027. The engine definesformula_2025, so the export guard drops it from the person table at 2026+. Documented to the partner already; make sure schema docs list it.- Schema documentation emitter. Release contract files today (
build_manifest,calibration_diagnostics,us_source_coverage,gate_summary,held_back_columns, ...) include no column-level schema doc. Each delivery needs a table/column/dtype manifest plus a diff (additions, renames, removals) against the prior release; first known entry iswould_claim_wic→takes_up_wic_if_eligible.
Sequencing
- Rebase and land #745; regenerate the pooled base and multispine pool with the new columns.
- Build the county target surface and the 2023 geography crosswalk in parallel.
- Calibrate at county grain with per-county support/ESS diagnostics and multi-level gates; publish a validation report per county (record count, calibrated aggregates vs targets, deviation).
- Produce the 2026 file, then 2027 on projected targets.
- Derived-measures sidecar + schema manifest emitter; documentation of file layout.
Related: #719, #745, #696, #292, #367.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the sequencing section, then inspect packages/microcosm-calibrate/, us_runtime/puma_ladder.py, us_runtime/geography_ladder.py, and the two release tools named in the issue. Review #745 and its related issues before choosing a narrowly scoped component. Done requires the county targets, geography, period handling, materialized measures, schema manifest, calibration outputs, and 2026/2027 deliveries to satisfy the stated contract.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering, databases, documentation
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100