PolicyEngine / PolicyEngine/policybench
scenario_121 encodes disability in inputs PE does not map to USDA disability, zeroing SNAP
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2
- Forks
- 3
- Avg merge
- 12h 7m
- Merged PRs (30d)
- 13
Description
Problem
scenario_121 (SC, single 53-year-old) intends a disabled household: the prompt narrates disability, and the inputs set is_disabled: true plus disability_benefits: 13044. But neither input is USDA-qualifying disability under 7 CFR 271.2 (which is receipt-based: SSI, SSDI, VA disability, etc.):
disability_benefitsis PolicyEngine's employer/private disability income variable — not a qualifying federal programis_disabledis a bare boolean that PE reads for the SNAP work-registration exemption but not foris_usda_disabled
So the reference pipeline computes a non-disabled SNAP unit: the household fails the 130% gross-income test (ratio 2.55) and gets $0, and the $30k of medical expenses never unlocks the excess-medical deduction. Force is_usda_disabled=true and the same household is eligible with $286/mo ($3,457/yr): elderly/disabled units are exempt from the gross test, and net income collapses to ~$41 after the medical deduction.
The scenario is internally ill-posed rather than PE being wrong — the ground-truth household doesn't represent what the prompt describes. Full trace in PolicyEngine/policybench#91 (row 13).
Options for the next reference refresh
- Preferred: re-encode the disability as a receipt-based input PE consumes for
is_usda_disabled— e.g. route the disability income tosocial_security_disability(SSDI receipt, SGA-independent). Note the SSI route does not work here: $25,665 wages = $2,139/mo > the 2026 SGA limit. - Alternatively, drop the SNAP output for this scenario as a scope gap.
Either changes a frozen scenario/reference, so it belongs in the next snapshot/reference-refresh cycle, not a live patch. Models that treated the household as disabled-for-SNAP and answered a positive amount are currently scored wrong on this cell.
Also worth considering
A generator-side lint: any scenario whose prompt narrates disability should set at least one receipt-based disabled_programs input (is_ssi_disabled chain, social_security_disability, veteran flags), since is_disabled alone propagates inconsistently in policyengine-us (work-registration yes, gross-test exemption no).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing scenario_121 and the full trace in PolicyEngine/policybench#91, then inspect how the next snapshot/reference-refresh cycle stores scenario inputs and outputs. Confirm the receipt-based disability input and expected SNAP result against the cited USDA rule; done means the refreshed reference no longer scores the described disabled household as zero, or the SNAP output is removed if that scope gap remains.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100