PolicyEngine / PolicyEngine/microcosm

Geography ladder: use ASEC identified county, adopt in the stacked path, declare the block-draw conditioning

Open
#696 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Max, 2026-08-15: "each spine record should assign a census block pop-weighted-randomly within the finest geography they actually have — county or state for asec, puma for acs."

The machinery is close (block-anchored, pop-weighted throughout) but has two gaps and one undeclared decision:

What runs today

  • puma_ladder.py: ACS keeps its observed PUMA; ASEC gets a PUMA sampled within state ∝ 2020 PUMA population; CD sampled within PUMA ∝ block-population overlap (119th, vintage-refusing).
  • geography_ladder.py: one 2020 tabulation block per household, sampled within the assigned CD ∝ block population; tract/county = structural prefixes of the block; place/SLD/CBSA from block crosswalks.
  • Both run in tools/build_us_acs_local_release.py (dense-era #275 path).

Gap 1: ASEC identified-county is ignored

ASEC publishes the county code for identified counties (~half the population). The ladder treats every ASEC record as state-only. Fix: when the ASEC record carries an identified county, sample the PUMA ∝ the (PUMA, county) block-population overlap restricted to that county (the overlap table already exists in the ladder artifact); fall back to state otherwise. "Finest geography they actually have" becomes the invariant, and the lineage spec should declare it per channel.

Gap 2: the stacked path doesn't run the ladder at all

Stacked pool artifacts (f010/f025 verified) carry only state_fips + PUMA. The block/CD/county assignment lives in the dense-era local-release tool and hasn't been adopted by the stacked pipeline. Decide the mount point (publish step vs filter-time per #275) and wire it in — CD-in-every-file-size (Max's 8/15 calibration ruling on #624) needs CD assigned on every artifact.

Undeclared decision: block draw conditions on CD, not CD ∩ PUMA

A record's block can land in a different PUMA of the same district than the PUMA it carries (final geography stays self-consistent because county/tract derive from the block; the PUMA becomes a stepping stone). Either declare that intentional in the module docstring + lineage spec, or condition the block draw on the (CD, PUMA) intersection.

Receipts: per-CD support at f010/f025 measured in #624 (comment 5303751580); geography rows should join the lineage spec (specs/us_imputation_lineage.yaml, #695) as their own class — assigned-by-ladder, distinct from measured/imputed/computed.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with puma_ladder.py, geography_ladder.py, and tools/build_us_acs_local_release.py to trace the existing dense-era assignment and compare it with the stacked f010/f025 publish path. Read specs/us_imputation_lineage.yaml and the receipts from #624 and #695 before deciding where the ladder belongs and how the CD/PUMA condition is declared. Done means identified-county ASEC handling, stacked-artifact adoption, CD assignment in every file size, and lineage coverage are implemented and documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.