PolicyEngine / PolicyEngine/snap-qc-sim

Cross-check Bauer–Schanzenbach Appendix B against the case-level bootstrap (FY2023)

Open
#89 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
0
Avg merge
18m
Merged PRs (30d)
5

Description

Twice-offered to Diane Schanzenbach (accepted in spirit — "happy to provide feedback", 8/27). The calibration handshake between the two teams' machinery.

  1. Extract the published per-state FY2023 bin probabilities and standard errors from Bauer–Schanzenbach 2026 (Hamilton Project, "The SNAP state cost-shift policy leaves the program's existence to chance", Appendix B + Table 27 of the FY2023 QC Annual Report they source). Hash the acquisition per the issuance-file convention (analysis/issuance_fy2024.json source block is the model).
  2. Replicate their construction exactly: Normal(point, official SE), their bins including the 13.33% carveout treatment.
  3. Run our case-level bootstrap on the FY2023 raw file (already in the audited cache — see analysis/coding_consistency.json years) at the FY2023 official threshold, anchored on FY2023 officials.
  4. Compare per state: bin probabilities (total-variation distance), SD scale (their design SEs vs our i.i.d. bootstrap — expect ours smaller; the earlier national comparison was ~0.8 vs ~1.1pp), tail-shape divergence. Report agreement AND divergence honestly — the earlier bootstrap-vs-normal check at FY2024 found shape effects second-order (median TV 0.005, max 0.046), so scale will likely dominate.

Output: artifact + memo + locks per repo conventions. This is the first deliverable of the Diane feedback loop and should be shareable with her directly.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the source block in analysis/issuance_fy2024.json and the FY2023 year references in analysis/coding_consistency.json, then locate the audited FY2023 raw file. Follow the stated bin, bootstrap, and comparison steps. Done means producing the artifact, memo, and locks required by repository conventions, with agreement and divergence reported per state.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
analytics, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.