PolicyEngine / PolicyEngine/microcosm

Stop SNAP state saturation from passing release and publish a person-compatible validation

Open
#647 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

From scorecard diagnosis batch 1, item #9 (classification: pe_gap, confidence high; full memo: PolicyEngine/policyengine-scorecard diagnosis/DIAGNOSES.md).

Divergence

AK has 107,166.399 PE eligible persons, 107,163.924 mapped participant persons, a 99.99769% rate, and a 2.476-person gap; Urban reports 86,000 eligible, 34.2%, and about 57,000 gap. The FY2024 FNS Alaska household target is hit within 6.495 households (0.0207%), yet the person statistic nearly saturates. Fifteen jurisdictions are exactly 100% (AL, AR, DC, HI, ID, IN, LA, MS, MO, ND, OK, RI, TN, UT, WY); AK, SD, and TX bring the >=99.9% count to 18. The release-stage pre-calibration diagnostic marks 30 states saturated but marks AK false, whereas AK's final-weight person rate is 99.998%; this establishes that the current stage diagnostic does not describe the final-weight person result, but post-calibration eligible-unit weights are still needed to identify AK's exact causal path.

Proposed fix

Build P accepts target-at-or-above-eligible SPM-unit weight as saturation, and its stage diagnostic uses pre-calibration weights. AK nevertheless ships as 99.998% of eligible persons although its FY2024 FNS household target is hit within 0.021%; the pre-calibration diagnostic itself marks AK unsaturated. Make unexplained final-weight saturation release-blocking; emit final-calibrated-weight eligible and caseload diagnostics; and add AK/systemic-saturation regression tests. In the scorecard, do not translate household-targeted SPM-unit receipt into an administrative-shaped person numerator by counting every member; add a SNAP assistance-unit/person counterpart or a separate FNS average-month-person validation.

Evidence

  • c80b6c4:data/comparison.json, Alaska SNAP: eligible_count is held_out; participation_rate is consumed_as_target through FNS state average-month household counts. A divergence on the consumed quantity is therefore a calibration-machinery finding.
  • Build P us_snap_state_take_up.json: the pre-calibration Alaska FY2024 target is 31,319.25 households, eligible weight 33,205.856, caseload 31,109.222, and saturated=false; the diagnostic explicitly uses pre-calibration design weights.
  • Build P calibration_diagnostics.json, usda_snap.fy2024.state_average_monthly_households.wro.ak.average_monthly_households@2024: 31,319.25 target versus 31,325.745 final (+0.0207%), so the official household target is internally consistent and hit.
  • POP@8828dee:packages/populace-build/src/populace/build/us_runtime/snap_state_take_up.py:186-211 derives target / weighted eligible and clips at one; lines 307-380 define saturation and document pre-calibration weights.
  • POP@8828dee:packages/populace-build/src/populace/build/us/source_stages.json:1806 states that saturated states set all eligible units to take-up and that person targets are deliberately absent because SPM-unit membership overcounts SNAP assistance-unit participants.
  • pipeline/compute_counterparts.py:300-306 nevertheless maps SPM-unit receipt and eligibility to every person for the scorecard.
  • c80b6c4:data/comparison.json, state SNAP total rows: 15 jurisdictions are exactly 100%; 18 are at least 99.9% after adding AK, SD, and TX.

🤖 Generated with Claude Code

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with packages/populace-build/src/populace/build/us_runtime/snap_state_take_up.py:186-211 and 307-380 to understand the existing saturation diagnostic and its pre-calibration inputs. Then read packages/populace-build/src/populace/build/us/source_stages.json:1806 and pipeline/compute_counterparts.py:300-306, using the cited Build P and scorecard evidence. Done means final-weight saturation is release-blocking, eligible and caseload diagnostics are emitted, and AK/systemic-saturation and person-compatible validation regressions cover the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, release, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.