PolicyEngine / PolicyEngine/microcosm
Early Head Start ships at engine-default 100% take-up: $112.3B (≈30× ACF spending) inside household_net_income on the certified 2024 artifact; CHIP at 100% is program-aggregate-only
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 4
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 94
Description
Observed on the certified artifact
Certified release populace-us-2024-buildp-sparse-rmloss100-cae8640-20260728T011454Z, loaded via policyengine.py 5.0.1 managed_microsimulation (engine policyengine-us 1.764.6, period 2024), weighted sums (2026-08-01 take-up mechanics audit):
| program | weighted participants | benefit dollars | take-up among modeled eligibles | flag stored in artifact |
|---|---|---|---|---|
early_head_start |
5,155,497 | $112.256B | 100% (5.16M of 5.16M) | no |
chip |
4,322,070 | $12.977B | 100% | no |
head_start (working contrast) |
295,215 | $4.903B | 5.9% of 5.03M | yes (SIPP-seeded) |
The artifact stores nine takes_up_* columns (aca, eitc, head_start, housing_assistance, medicaid, medicare, snap, ssi, tanf). takes_up_early_head_start_if_eligible and takes_up_chip_if_eligible are not among them, so the engine's default_value = True applies to every record — the "mechanical 100% participation" the take-up contract's own doctrine text predicts for unseeded data_seeded leaves.
Administrative anchors
- The engine's own
gov.hhs.head_start.early_head_startparameters (2023-09 vintage, summed across states): $3.796B spending / 180,873 enrollment (≈$21.0k per enrollee). The artifact distributes ≈29.6× the engine's own spending parameter. - ACF/OHS PIR service snapshot 2023–24 (the same PDF pinned in
test_us_early_head_start_exclusion.py): 187,966 funded slots; 232,842 cumulative enrollment (219,916 children + 12,926 pregnant women). The artifact's 5.16M weighted participants are ≈22–27× either count. - CHIP: the engine's separate-CHIP calibration parameters (2024) sum to 3,341,711 enrollment / $8.394B spending; the artifact computes 4,322,070 / $12.977B (≈1.3× / 1.5×). Much milder, and the fix path is already adjudicated (below).
Mechanism (each step read in code at policyengine-us 1.764.6, commit 92e6052d3e)
takes_up_early_head_start_if_eligibleandtakes_up_chip_if_eligibleare bare inputs,default_value = True, no formula.early_head_startpays statespending / enrollment(≈$21.8k average across the artifact's takers) to every taker;is_early_head_start_eligible= (age < 3 or pregnant) and (Head Start income-eligible or categorically eligible) → 5.16M weighted persons.chippaysper_capita_chip(separate-CHIP state spending / enrollment) gated bychip_enrolled = is_chip_eligible & takes_up_chip_if_eligible.
This is not a build-process failure: the take-up contract deliberately records both flags as rate_unsourced ("left unseeded", #313 parity-gate exemption), and the EHS evidence contract (test_us_early_head_start_exclusion.py, ecps_parity_known_gaps.json) proves the source unavailability. The problem is the composition: adjudicated non-seeding + engine default True + downstream aggregation = a ~30× phantom on the certified surface.
Downstream exposure, verified at 1.764.6
household_net_incomeis contaminated.early_head_startandhead_startare in thegov.household.household_benefitsparameter list (2022 and 2024 brackets) →household_benefits→household_net_income.adds. ≈$112B of phantom benefits, concentrated at ≈$21.8k/person on households with an under-3 child or pregnant member inside the Head Start eligibility domain. Anything computed fromhousehold_benefitsorhousehold_net_income(benefit aggregates, net-income levels and decile outputs) inherits it.- SPM poverty is clean. Neither program (nor chip/medicaid) appears in
spm_unit_benefits, andin_poverty ← poverty_gap ← spm_unit_net_income ← spm_unit_benefits. Poverty baselines are unaffected. - CHIP is program-aggregate-only by default.
chipsits inhousehold_health_benefits, which returns 0 unlessgov.simulation.include_health_benefits_in_net_income(default false). Exposure is thechipaggregate itself plus the health-inclusive net-income variants. - Weights are unaffected. The fiscal-targets registry (
fiscal_targets.py) and target-parity manifest contain no head_start reference and nohousehold_benefitsexpression, so EHS never enters calibration; the distortion is purely output-side. (CHIP enrollment targets do exist and carry the concept mismatch tracked in #321.)
Why the obvious data-side fixes are already foreclosed
- A sourced admin rate: the contract's provenance rule requires an administrative participation rate; ACF publishes none. PIR cumulative enrollment (232,842) includes turnover; funded enrollment (187,966) is capacity. Both are pinned as "semantic non-substitutes" in
test_us_early_head_start_exclusion.py, and the retired us-data 9% NIEER scalar is the fabricated-rate class ledger#77 removed. PIR enrollment counts therefore do not clear the bar for a rate-seeded stage. - A count_calibrated stage: that treatment (medicaid pattern, #331) requires a survey-reported anchor column, and the evidence contract proves none exists for EHS in any locked input (SIPP
EEDHEADSThas zero under-3 positives, the child-care items begin at age 2, there is no pregnancy branch; ASEC/PUF carry no enrollment signal). - CHIP: already adjudicated — blocked on the #321 M-CHIP/separate-CHIP concept split, then follows the #331 pattern. Its numbers above are for the record, not a new ask.
Options
- Engine-side gate (recommended). policyengine-us moves
head_start/early_head_startout of the always-onhousehold_benefitslist or behind an include-switch — the exact precedent used for medicaid/msp/chip/aca (household_health_benefitsbehindgov.simulation.include_health_benefits_in_net_income, default false). Fixes any dataset that does not store the flag, and preserves per-program calculator display. Companion issue: PolicyEngine/policyengine-us#9192. Note this only cleans the headline surface —early_head_startitself remains a 100%-take-up aggregate until a source exists, so aggregate consumers still must not sum it. - Engine default flip for these flags: conflicts with the household-calculator contract (default True means "show what an enrollee would get"); not recommended.
- New populace treatment class ("unrepresentable program": store all-False with the same evidence): converts the +$108B error into −$3.8B, but is a doctrine change —
rate_unsourcedtoday deliberately means "leave the engine default", and all-False is as unsourced as any other rate. Needs explicit adjudication if chosen. - Status quo + disclosure: keep the parity-gap exemption and document the certified-surface contamination in release notes so consumers don't sum
early_head_start/chipor readhousehold_net_incomelevels as calibrated.
Adjacent, tracked elsewhere: WIC at 1.764.6 runs its own machinery (52.8% of eligibles on this artifact; no takes-up flag consumed) and the 1.779.4 takes_up_wic_if_eligible contract drift belongs to the #324 pin-bump lane. takes_up_basic_health_program_if_eligible and takes_up_dc_ptc are the same rate_unsourced class (no numbers recorded here). #591 tracks the separate default-value-hole class (is_tanf_enrolled etc.).
Repro: weighted sums of early_head_start, chip, head_start and their eligibility variables on the certified artifact via managed_microsimulation (period 2024); engine wiring verified with git show 92e6052d3e:<path> against policyengine-us (household_benefits.yaml, spm_unit_benefits.py, household_net_income.py, household_health_benefits.{py,yaml}, include_health_benefits_in_net_income.yaml, the early_head_start/chip variable files).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the weighted sums with managed_microsimulation for the certified 2024 artifact, then read the named policyengine-us files, especially household_benefits.yaml, household_health_benefits.py, and household_net_income.py. Compare the proposed engine-side gate with the documented alternatives and companion issue #9192. Done requires an adjudicated treatment that prevents the contaminated household_net_income surface while preserving the stated contract and documenting remaining aggregate limitations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, data
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100