PolicyEngine / PolicyEngine/policybench

v2 prompt and data: program-specific disability facts, integer inputs, and the take-up boundary

Open
#165 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2
Forks
3
Avg merge
12h 7m
Merged PRs (30d)
13

Description

v2 prompt and data: make disability facts program-specific, expose integer inputs, state the take-up boundary

Problem (found on the 39-model board, 2026-09-05)

The prompt lists a single undefined fact, - is disabled, and tells models that unlisted status facts are false and to assume program take-up. The engine's disability-linked gates are program-specific and read other inputs:

Program Engine gate Present in the certified June build?
SSI meets_ssi_disability_criteria and the SGA screen (42 USC 1382c) false for all 160,858 persons; never appears in any prompt
Medicare under 65 social_security_disability > 0 and months_receiving_social_security_disability >= 24 (42 USC 426) months absent in every build, and integer-typed, which the prompt builder excludes
SNAP elderly/disabled receipt-based is_usda_disabled works (SSI-disabled, SSDI income, veteran statuses)
IRC 152 / 22 is_permanently_and_totally_disabled never populated in any build
IRC 21 CDCC is_incapable_of_self_care absent until the 2026-07-23 builds (2,067 from PEDISDRS)

is_disabled itself (any of six CPS difficulty items) feeds 128 variables: 113 state child-care, TANF and state-tax provisions, HUD, Medicaid work-requirement exemptions, VITA, and the SSI working-student exclusion (which should read is_ssi_disabled).

Consequences on the 39-model board: 34 SSI model-rows across five households paid SSI on a disabled under-65 member against a $0 reference fixed by the data; 18 of them are pure disability-gate disagreements (within 10% of the clean alternative amount), 16 carry independent errors (resource limit exceeded, one-third reduction from inferred shelter, wrong rates). One Medicare row (scenario_074) rests on the unlisted SSDI duration. See annotations//us_adjudications.json and the paper's audit section.

The preamble also does not state the take-up boundary: the engine deems the TANF non-cash benefit received when eligible (SNAP BBCE) while models are told not to infer unlisted benefit receipt.

Proposed preamble (Astra draft, 2026-09-05, to edit)

Estimate the requested outputs under the PolicyEngine-US version and calculation conventions identified with this question. All listed people live together in the stated household group. Unless dated explicitly, facts apply throughout the tax-benefit year without changes in status or income volatility; wage amounts are annual totals including overtime, and hourly rates are straight-time rates. Unlisted numeric inputs, including integer inputs, are zero, and unlisted boolean inputs are false, subject to the filing and take-up assumptions below. "General disability indicator" is a survey characteristic and does not establish SSI disability, tax-specific disability, SSDI entitlement, or Medicare eligibility. SSI disability uses the separately stated "meets SSI disability criteria before the substantial-gainful-activity test"; apply the earnings test separately. Tax-specific disability and self-care conditions are also stated separately. Medicare eligibility is evaluated on January 1; qualifying disability-benefit months are measured as of that date. Assume filing and full take-up of eligible requested benefits, plus eligible TANF/MOE noncash benefits used for SNAP categorical eligibility, and apply those computed benefits in downstream calculations. This exception permits computed receipt; it does not create substantive eligibility or historical entitlement. Social Security, SSDI, and veterans payments and histories are supplied inputs only. Infer no other income, expenses, assets, receipt, rent, or coverage. The SSI output includes federal SSI only; state supplements require a separate requested output.

Data and builder requirements

  • Draw v2 households from a microcosm build that carries the SIPP-imputed meets_ssi_disability_criteria (builds from 2026-07-17) and is_incapable_of_self_care (from 2026-07-23).
  • Populate is_permanently_and_totally_disabled (or state that IRC 152/22 disability pathways are out of scope).
  • Carry SSDI entitlement duration (or start date) with a defined reference date, or state the assumption; allow integer inputs to be promptable.
  • Label employer_sponsored_insurance_premiums as employer-paid (never in the stated wages).
  • Record provenance per disability fact: observed, imputed, or unknown. Unknown must not print as an observed negative.
  • Engine nit: is_ssi_blind_or_disabled_working_student_exclusion_eligible reads is_disabled; it should read is_ssi_disabled.

Related: #139 (v2 board condition), the prompt-ambiguity adjudication record introduced in #164.

Additions from the 2026-09-05 work (PR #164)

  • The prompt builder exposes only float and boolean inputs; months_receiving_social_security_disability is integer-typed and can never appear. Make integer inputs promptable.
  • Scenario.to_pe_household() defined no marital units; policyengine-core lumped every member into one, deeming the whole household's income to any SSI-eligible adult. Fixed in #164 (head and spouse as the only couple); v2 should carry person_marital_unit_id from the data instead of inferring it.
  • Regenerating today's references on policyengine-us 1.822.5 moves 36 of 1,984 outputs; one change (weekly hours default 0 instead of 40, so missing hours fail the SNAP work-requirement test) turns SNAP into an unlisted-input dependency for any household whose prompt lists no hours. v2 must list weekly hours whenever the engine reads them, or state the default in the preamble.
  • Standing gate for v2 and every later board: recompute every reference under each unlisted-input reading before scoring; exclude what moves (the reference_exclusions.json mechanism from #164), rather than waiting for a judge to notice.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the prompt builder, Scenario.to_pe_household(), annotations//us_adjudications.json, and the reference_exclusions.json mechanism introduced in #164. Trace how integer inputs, disability facts, marital units, and weekly hours reach prompts and references. Done means the stated v2 prompt and data requirements are implemented or explicitly scoped out, provenance and exclusions are recorded, and affected references are recomputed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.