PolicyEngine / PolicyEngine/microcosm

SNAP payment-error layer: full-universe microsimulation of QC-measured error

Open
#668 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Claiming the surface for the end state described in snap-qc-sim/analysis/MICROSIM_ROADMAP.md: payment-error simulation as a proper microsimulation on the Microcosm caseload, replacing the deployed simulator's resample-of-a-sample scaffold.

Architecture (two-institute split)

  • Deterministic layer (Axiom): the rules engine computes what should happen per case — already exact against the recorded benefit chains of seven states' FY2024 QC files (zero tolerance, 6,081/6,081 in-scope cases), served through the policyengine.py interface as encodings migrate to rulespec.
  • Estimation layer (this repo's side): a case-level model of what does happen — deviation incidence, size, and element mix as functions of case characteristics and state policy features. Payment error = the gap between layers.
  • Measurement layer: QC measurement modeled as what it is — a designed sample drawn from the universe with thresholds and review — so sampling noise, sample-size levers, and review-practice variation become derived properties.

The QC public-use files then serve as verification oracle (rules side) and calibration/validation data (error side), not as the simulation universe. The official-vs-file "wedge" (federal re-review integration + ineligible-case error; median 31% of the official FY2024 rate per state, up to 81% for AK — quantified and test-locked in snap-qc-sim) becomes model output calibrated to the official rate's published components instead of a fixed offset. The error layer stays a modeled overlay at analysis time — like take-up — never baked into certified raw data.

Rungs (from the memo)

  1. (shipped, snap-qc-sim) Resample app with the wedge disclosed per state.
  2. Official-component calibration-target registry (per-state overpayment / underpayment / ineligible-case, QC annual reports, hash-registry discipline).
  3. Colorado-first prototype: Microcosm cases × engine benefits × error-model deviations, calibrated to official components, validated against the realized FY2024→25 movement and the QC file's own moments.
  4. Measurement simulator on the prototype universe.
  5. Graduation into this repository behind its certification gates.

Critical path

The error model is not yet load-bearing: distributional coverage fails (all nine gaps one-sided negative), cross-state level gaps persist, seven jurisdictions gated. Partner-state administrative data is the most promising unlock; additional file years the second. Until that clears, rungs 2 and the imputation groundwork (QC-grade case detail — deduction composition, certification timing, BBCE status — imputed onto Microcosm units, trained from the QC files) can proceed in parallel.

Context: deployed simulator at policyengine.org/us/snap-payment-error-simulator; working paper at /paper on the same route; quasi-experimental designs that could put design-based parameters into the estimation layer in snap-qc-sim/analysis/QUASI_EXPERIMENTS.md.

🤖 Generated with Claude Code

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with analysis/MICROSIM_ROADMAP.md and analysis/QUASI_EXPERIMENTS.md, then inspect the deployed simulator route and the Microcosm caseload context. The issue describes a multi-rung architecture rather than a specific entry point or test. Done would mean a validated Colorado-first prototype progressing toward measurement simulation and eventual graduation behind certification gates.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
18/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.