PolicyEngine / PolicyEngine/microcosm

Early Head Start ships at engine-default 100% take-up: $112.3B (≈30× ACF spending) inside household_net_income on the certified 2024 artifact; CHIP at 100% is program-aggregate-only

Open
#593 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Observed on the certified artifact

Certified release populace-us-2024-buildp-sparse-rmloss100-cae8640-20260728T011454Z, loaded via policyengine.py 5.0.1 managed_microsimulation (engine policyengine-us 1.764.6, period 2024), weighted sums (2026-08-01 take-up mechanics audit):

program weighted participants benefit dollars take-up among modeled eligibles flag stored in artifact
early_head_start 5,155,497 $112.256B 100% (5.16M of 5.16M) no
chip 4,322,070 $12.977B 100% no
head_start (working contrast) 295,215 $4.903B 5.9% of 5.03M yes (SIPP-seeded)

The artifact stores nine takes_up_* columns (aca, eitc, head_start, housing_assistance, medicaid, medicare, snap, ssi, tanf). takes_up_early_head_start_if_eligible and takes_up_chip_if_eligible are not among them, so the engine's default_value = True applies to every record — the "mechanical 100% participation" the take-up contract's own doctrine text predicts for unseeded data_seeded leaves.

Administrative anchors

  • The engine's own gov.hhs.head_start.early_head_start parameters (2023-09 vintage, summed across states): $3.796B spending / 180,873 enrollment (≈$21.0k per enrollee). The artifact distributes ≈29.6× the engine's own spending parameter.
  • ACF/OHS PIR service snapshot 2023–24 (the same PDF pinned in test_us_early_head_start_exclusion.py): 187,966 funded slots; 232,842 cumulative enrollment (219,916 children + 12,926 pregnant women). The artifact's 5.16M weighted participants are ≈22–27× either count.
  • CHIP: the engine's separate-CHIP calibration parameters (2024) sum to 3,341,711 enrollment / $8.394B spending; the artifact computes 4,322,070 / $12.977B (≈1.3× / 1.5×). Much milder, and the fix path is already adjudicated (below).

Mechanism (each step read in code at policyengine-us 1.764.6, commit 92e6052d3e)

  • takes_up_early_head_start_if_eligible and takes_up_chip_if_eligible are bare inputs, default_value = True, no formula.
  • early_head_start pays state spending / enrollment (≈$21.8k average across the artifact's takers) to every taker; is_early_head_start_eligible = (age < 3 or pregnant) and (Head Start income-eligible or categorically eligible) → 5.16M weighted persons.
  • chip pays per_capita_chip (separate-CHIP state spending / enrollment) gated by chip_enrolled = is_chip_eligible & takes_up_chip_if_eligible.

This is not a build-process failure: the take-up contract deliberately records both flags as rate_unsourced ("left unseeded", #313 parity-gate exemption), and the EHS evidence contract (test_us_early_head_start_exclusion.py, ecps_parity_known_gaps.json) proves the source unavailability. The problem is the composition: adjudicated non-seeding + engine default True + downstream aggregation = a ~30× phantom on the certified surface.

Downstream exposure, verified at 1.764.6

  • household_net_income is contaminated. early_head_start and head_start are in the gov.household.household_benefits parameter list (2022 and 2024 brackets) → household_benefitshousehold_net_income.adds. ≈$112B of phantom benefits, concentrated at ≈$21.8k/person on households with an under-3 child or pregnant member inside the Head Start eligibility domain. Anything computed from household_benefits or household_net_income (benefit aggregates, net-income levels and decile outputs) inherits it.
  • SPM poverty is clean. Neither program (nor chip/medicaid) appears in spm_unit_benefits, and in_poverty ← poverty_gap ← spm_unit_net_income ← spm_unit_benefits. Poverty baselines are unaffected.
  • CHIP is program-aggregate-only by default. chip sits in household_health_benefits, which returns 0 unless gov.simulation.include_health_benefits_in_net_income (default false). Exposure is the chip aggregate itself plus the health-inclusive net-income variants.
  • Weights are unaffected. The fiscal-targets registry (fiscal_targets.py) and target-parity manifest contain no head_start reference and no household_benefits expression, so EHS never enters calibration; the distortion is purely output-side. (CHIP enrollment targets do exist and carry the concept mismatch tracked in #321.)

Why the obvious data-side fixes are already foreclosed

  • A sourced admin rate: the contract's provenance rule requires an administrative participation rate; ACF publishes none. PIR cumulative enrollment (232,842) includes turnover; funded enrollment (187,966) is capacity. Both are pinned as "semantic non-substitutes" in test_us_early_head_start_exclusion.py, and the retired us-data 9% NIEER scalar is the fabricated-rate class ledger#77 removed. PIR enrollment counts therefore do not clear the bar for a rate-seeded stage.
  • A count_calibrated stage: that treatment (medicaid pattern, #331) requires a survey-reported anchor column, and the evidence contract proves none exists for EHS in any locked input (SIPP EEDHEADST has zero under-3 positives, the child-care items begin at age 2, there is no pregnancy branch; ASEC/PUF carry no enrollment signal).
  • CHIP: already adjudicated — blocked on the #321 M-CHIP/separate-CHIP concept split, then follows the #331 pattern. Its numbers above are for the record, not a new ask.

Options

  1. Engine-side gate (recommended). policyengine-us moves head_start/early_head_start out of the always-on household_benefits list or behind an include-switch — the exact precedent used for medicaid/msp/chip/aca (household_health_benefits behind gov.simulation.include_health_benefits_in_net_income, default false). Fixes any dataset that does not store the flag, and preserves per-program calculator display. Companion issue: PolicyEngine/policyengine-us#9192. Note this only cleans the headline surface — early_head_start itself remains a 100%-take-up aggregate until a source exists, so aggregate consumers still must not sum it.
  2. Engine default flip for these flags: conflicts with the household-calculator contract (default True means "show what an enrollee would get"); not recommended.
  3. New populace treatment class ("unrepresentable program": store all-False with the same evidence): converts the +$108B error into −$3.8B, but is a doctrine change — rate_unsourced today deliberately means "leave the engine default", and all-False is as unsourced as any other rate. Needs explicit adjudication if chosen.
  4. Status quo + disclosure: keep the parity-gap exemption and document the certified-surface contamination in release notes so consumers don't sum early_head_start/chip or read household_net_income levels as calibrated.

Adjacent, tracked elsewhere: WIC at 1.764.6 runs its own machinery (52.8% of eligibles on this artifact; no takes-up flag consumed) and the 1.779.4 takes_up_wic_if_eligible contract drift belongs to the #324 pin-bump lane. takes_up_basic_health_program_if_eligible and takes_up_dc_ptc are the same rate_unsourced class (no numbers recorded here). #591 tracks the separate default-value-hole class (is_tanf_enrolled etc.).

Repro: weighted sums of early_head_start, chip, head_start and their eligibility variables on the certified artifact via managed_microsimulation (period 2024); engine wiring verified with git show 92e6052d3e:<path> against policyengine-us (household_benefits.yaml, spm_unit_benefits.py, household_net_income.py, household_health_benefits.{py,yaml}, include_health_benefits_in_net_income.yaml, the early_head_start/chip variable files).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the weighted sums with managed_microsimulation for the certified 2024 artifact, then read the named policyengine-us files, especially household_benefits.yaml, household_health_benefits.py, and household_net_income.py. Compare the proposed engine-side gate with the documented alternatives and companion issue #9192. Done requires an adjudicated treatment that prevents the contaminated household_net_income surface while preserving the stated contract and documenting remaining aggregate limitations.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, data
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.