PolicyEngine / PolicyEngine/microcosm

Diagnose the child-vs-total SPM composition anomaly (poverty stays a permanent holdout)

Open
#646 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

From scorecard diagnosis batch 1, item #8 (classification: pe_gap, confidence high; full memo: PolicyEngine/policyengine-scorecard diagnosis/DIAGNOSES.md).

Divergence

PE exceeds Urban by 3.1689 percentage points (22.96% relative). More decisively, PE exceeds the same-year Census 2024 child SPM benchmark by 3.5689 points (26.63%), while its total rate exceeds Census by only 0.7042 points. Urban's 2023 13.8% is only 0.4 point above the separate Census 2024 child benchmark despite the year mismatch. The age-specific pattern and same-year benchmark reject a general 2023-to-2024 shift as the main explanation. EHS contributes exactly zero to the scorecard SPM calculation.

Proposed fix

Build P's 2024 child SPM poverty rate is 16.97% versus Census P60-287's same-year 13.4%, even though total poverty is much closer (13.60% versus 12.9%). Promote the existing national total and child P60-287 rows from passive backtests to explicit release diagnostics with thresholds. Add child-versus-adult decompositions of market income, cash and in-kind benefits, taxes, medical expenses, work/child-care expenses, and SPM thresholds. Poverty rates are PERMANENT calibration holdouts (doctrine 2026-08-02): use these comparisons to locate the child-specific resource or threshold error; fitting poverty directly is prohibited, and a deviation from the official rate is not itself the defect — the child-vs-total asymmetry is what warrants investigation.

Evidence

  • c80b6c4:data/comparison.json, child SPM poverty_rate: held_out with no Build P poverty targets; PE 16.9689% versus Urban 13.8%.
  • POP@8828dee:packages/populace-build/src/populace/build/us/state_spm_poverty_levels.json:1-32 records Census P60-287 single-year 2024 benchmarks of 13.4% for children under 18 and 12.9% overall; these are out-of-sample.
  • data/pe/pe_metrics.json: PE 2024 child poor count 13,012,226.578 / population 76,683,005.281 = 16.9689%; total is 46,264,651.701 / 340,077,321.941 = 13.6042%.
  • PEUS@92e6052d3e spm_unit_is_in_spm_poverty.py:10-13 compares spm_unit_net_income with spm_unit_spm_threshold; in_poverty.py:11-12 and poverty_gap.py:12-15 implement the same strict comparison used by the Census backtest; spm_unit_net_income.py:11-18 adds market income and benefits and subtracts taxes and SPM expenses.
  • docs/replication-assessment.md:19,130-133 documents PE annual 2024 versus ATTIS average-month 2023 on pooled 2022+2023 ACS.
  • PEUS@92e6052d3e:policyengine_us/variables/household/income/spm_unit/spm_unit_benefits.py:11-84 omits Head Start and Early Head Start, ruling out the $112.3B EHS valuation defect as a cause of this SPM rate.

Adjudication note

Adjudicator amendment (Max, 2026-08-02): (1) poverty metrics are PERMANENT calibration holdouts — never targets, not 'held out initially'; targets come from administrative systems only. (2) Deviations from official poverty statistics are never inherently problematic — populace corrects survey benefit underreporting (admin-calibrated benefit dollars exceed survey-reported amounts), so sitting below survey-based rates is the expected direction, all else equal. What flags THIS item is the anomaly: the child rate sits ABOVE Census while the total sits near it. Investigate composition; the deviation itself is not the defect and the official number is a comparator, not truth.

🤖 Generated with Claude Code

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with data/comparison.json and the referenced SPM variables, especially spm_unit_net_income.py, spm_unit_is_in_spm_poverty.py, in_poverty.py, and poverty_gap.py. Trace the child-versus-total resource and threshold components, using the Census benchmarks in state_spm_poverty_levels.json as comparison data. Done means explicit release diagnostics and decompositions identify the child-specific asymmetry without fitting poverty metrics as targets.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
analytics, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.