Sparse-57k export drops 94 no-formula input columns from the dense parent (tips, overtime, disability flags, ESI premiums, rent, QBI basis)

Summary

The sparse-57k export (populace-us-2024-sparse-l0-refit-57k-71a0887, certified default for bundles 4.18.8 and 4.20.0) drops 94 no-formula PolicyEngine-US input columns that the dense parent (f0af251, 4.18.7 default) populates. #278 documented mass-zeroing for a handful of bases and #356 the three SSI asset columns; this is the full inventory of the column-presence regression. The Build-H dense parent published today (populace-us-2024-buildh-dense-warmstart-b449eb7-20260708T100442Z) drops essentially the same set (129 of 133 columns), so the Build-I successor (#359) inherits the gap unless the export carries them.

Method: person-table column diff (dense 249 cols → sparse 222), filtered to pe-us 1.752.2 variables with no formula and no adds/subtracts (i.e., true inputs; formula-owned exports like employment_income/social_security are correctly dropped per #24 and excluded here). Mass = dense column × household weight, TY2024. Full list + per-column mass attached below.

Dollar inputs lost (default 0 → mass vanishes; selected, dense-weighted)

Flag inputs lost with real signal (default False → population zeroed)

• is_disabled (12.6% weighted) — disability pathways in SSI/Medicaid, SNAP ABAWD exemptions (being actively seeded in #350–#353; those seeds vanish at export unless presence-gated)
• is_blind (3.1%) — SSI/Medicaid blind category, extra standard deduction
• is_pregnant (1.1%), receives_wic (0.8%) — WIC/Medicaid pregnancy pathways (#352)
• is_full_time_college_student (3.8%), is_separated, is_surviving_spouse, AOTC enrollment flags (~1.7%) — filing status and education credits
• immigration_status_str, ssn_card_type (categorical) — SSN/citizenship-conditioned policy (#225 family)

Currently-degenerate but contract-relevant (default True, stored all-true)

takes_up_ssi_if_eligible, takes_up_medicaid_if_eligible, takes_up_medicare_if_eligible, takes_up_{early_}head_start_if_eligible, would_claim_wic, is_wic_at_nutritional_risk, *_would_be_qualified QBI flags. Dropping these is a no-op today because the dense columns are all-true and the engine defaults match — but the moment real take-up rates are seeded (per the #312/#315 contract), the export must carry them or the seeds silently revert to full take-up.

Suggested fix

• Extend the Build-I export gate (#359 checks export mass on 35 columns) to the full input surface: column presence + mass vs the dense parent, with reviewed exclusions only for formula-owned exports (#24) and justified band-math cases.
• The selection step doesn't need new data — every dropped column exists on the pool rows; the subset export just needs to keep them.
• Classify each column in the #49 inventory (populated / upstream-populated / reviewed-excluded) as acceptance criteria.

Relates #278, #356, #359, #312, #315, #350, #352, #225, #38, #49. Found while re-running the SSI asset-limit analysis (PolicyEngine/ssi-asset-limits) on the certified small default via policyengine.py 4.20.0.

Start at the Build-I export gate in #359 and compare its current 35-column checks with the full input surface described here. Use the #49 inventory to classify columns, then verify sparse output presence and dense-weighted mass, excluding only formula-owned or reviewed band-math columns.