PolicyEngine / PolicyEngine/microcosm

Frozen eCPS artifacts ≥1.78.2 (incl. final 1.115.5) carry a misc-income point-mass defect that ~doubles tax aggregates — consumer advisory + release guard

Open
#457 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

tier:standard
Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Summary

The frozen Enhanced CPS artifacts on the Hugging Face model repo policyengine/policyengine-us-data carry a miscellaneous-income point-mass defect in every revision from 1.78.2 through the final 1.115.5 (the revision main resolves to). Any consumer running tax analysis against the frozen file's latest revisions gets roughly 2x-inflated income tax aggregates. Since the repo is deprecated and will not be re-cut, this needs a consumer advisory on the artifact itself plus a populace-side guard so the defect class can't recur in successor releases.

Evidence (measured on enhanced_cps_2024.h5)

Quantity rev 1.73.0 (Aug 2025) rev ≥1.78.2 (incl. 1.115.5)
miscellaneous_income, unweighted column sum $92B $2.07T
Largest single record $795M
Weighted AGI (pe-us 1.772.0, 2026) $15.7T (SOI-consistent, within 3%) $23.9T
Weighted federal income tax (2026) $2.43T (within 5% of SOI) $4.4T

The defect is the known QRF imputation landmine: quantile-regression-forest draws placing extreme point masses on miscellaneous_income for a handful of records. It entered at 1.78.2 and persists through the final frozen revision.

Discovered while pinning data for the IARIW OBBBA incidence paper (PolicyEngine/obbba-paper — private; cleaning manifest and revision comparison in data/clean/manifest.json and the methods section there). The paper pinned 1.73.0 and validated it against IRS SOI aggregates.

Asks

  1. Consumer advisory on the frozen artifact: a warning in the HF model card / README of policyengine/policyengine-us-data stating that revisions ≥1.78.2 (including main/1.115.5) should not be used for tax analysis, and pointing to 1.73.0 as the last SOI-consistent revision of that lineage (or to certified populace successors).
  2. Benchmark guard in populace release gates: a max-single-record / tail-mass check on imputed income columns (misc income being the proven offender) so a $795M single-record draw can never pass certification. Related work: #393 (dense-arm misc-income input-mass parity), #359 (selection-side misc-income support fix) — this issue is the artifact-level consumer warning + guard, complementary to those pipeline fixes.

Reproduction

from huggingface_hub import hf_hub_download
import h5py
for rev in ["1.73.0", "1.115.5"]:
    p = hf_hub_download("policyengine/policyengine-us-data",
                        "enhanced_cps_2024.h5", repo_type="model", revision=rev)
    with h5py.File(p) as f:
        col = f["miscellaneous_income"]["2024"][:]
        print(rev, f"sum=${col.sum()/1e9:,.0f}B", f"max=${col.max()/1e6:,.0f}M")

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the supplied Python reproduction against enhanced_cps_2024.h5 at revisions 1.73.0 and 1.115.5, then review the requested model-card/README warning and the populace release gates. Check related issues #393 and #359 for pipeline context. Done means the frozen artifact is clearly flagged with a safe revision or successor, and certification includes a max-single-record or tail-mass guard.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, release, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.