PolicyEngine / PolicyEngine/snap-qc-sim
Required-n: what sample size makes tier assignment reliable
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 18m
- Merged PRs (30d)
- 5
Description
The most decision-relevant number for the state-AG briefing Jonathan Meltzer is arranging (thread of 8/27), and the quantitative core of the 7 USC 2025(c)(1)(B) "valid statistical results" question. Also directly tests Bauer–Schanzenbach's asserted-but-not-analyzed claim that even CI-based approaches "will not solve" the problem. Strictly neutral: compute, don't recommend.
For each state: the QC sample size n* at which tier assignment reaches reliability targets (e.g., modal-tier probability ≥ 0.90/0.95 at the state's official rate), via the existing bootstrap machinery (resampling at m > n is already how the app prices extra audits — see simulate() extra_audits). Report:
- n* per state vs current sample size (and the ratio) at both sampling-only and persistence-widened dispersion (the honest forward-looking case; widened n* may be infinite for boundary states — that's a finding, not a failure: process variance doesn't shrink with n).
- The multi-year-averaging alternative: years of averaging k needed at current n (averaging cuts sampling variance ~1/k and transitory variance per the AR(1); the persistence layer's fitted parameters give the exact arithmetic).
- The CI-lower-bound alternative (pre-2014 practice per Bauer–Schanzenbach): billing frequency under lower-bound rules at current n.
The federal-subsample layer (7 CFR 275.3 — the subsample minimum that doesn't scale) is NOT modeled in the repo; state that as a scope boundary, do not encode reg mechanics without reading the reg text (no-fabricated-mechanisms).
Artifact + memo + locks; sol review before merge (external stakes — these numbers go to lawyers).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the existing simulate() bootstrap machinery, including the extra_audits path, and inspect how persistence parameters and current sample sizes are represented. Produce the per-state reliability, averaging, and CI-lower-bound results, plus the requested artifact, memo, and locks; explicitly document the federal-subsample layer as out of scope and obtain sol review before merge.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100