PolicyEngine / PolicyEngine/microcosm

Ingest ASEC 2024 and 2025 into the US support pool (spine is two vintages behind)

Open
#296 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

tier:standard
Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Problem

The US support spine pools raw Census ASEC vintages via build_pooled_asec_unit_frame (tools/build_us_asec_pooled_source_base.py takes --asec-h5 inputs), and the available ingested inputs stop at census_cps_2023.h5 (ASEC 2023, income year 2022). Census has since published:

  • ASEC 2024 (asecpub24csv.zip, income year 2023)
  • ASEC 2025 (asecpub25csv.zip, income year 2024) — released fall 2025, current latest

Production populace-us builds are therefore two survey vintages behind the newest available microdata.

Proposal

Ingest ASEC 2024 and 2025 into the raw-source lane (census_cps_2024.h5, census_cps_2025.h5) and move the pooled support spine's default year set forward so certified bundles build from the newest vintages (e.g. 2023–2025 pooled), with the usual population-share scaling.

Notes

  • The imputation-paper repo (github.com/PolicyEngine/imputation-paper) now reads ASEC 2024/2025 CSVs directly from census.gov (src/imputation_paper/data/cps.py); its loader documents the file conventions (PH_SEQ/H_SEQ join, two implied decimals on MARSUPWT/HSUP_WGT, zero-weight shell records in hhpub) and both bundles' column names verified compatible — useful as a reference for the ingestion.
  • Fixed sha256s: asecpub24csv.zip begins cdb39cda; column sets verified identical for the fields populace consumes.

🤖 Generated with Claude Code

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tools/build_us_asec_pooled_source_base.py and the build_pooled_asec_unit_frame entry point, then review the imputation-paper loader for ASEC 2024/2025 file conventions and joins. Ingest census_cps_2024.h5 and census_cps_2025.h5, update the pooled default year set, and verify that certified bundles build with the expected population-share scaling.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.