PolicyEngine / PolicyEngine/microcosm

US base v2: one CPS+ACS+PUF-detail pool; datasets labeled by exact record count (dense = full pool; exact-k L0 selection)

Open
#578 13 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

The acceptance rule (stated once, enforced everywhere)

A candidate release ships iff:

  1. It beats the incumbent on the frozen comparison register (same targets, both artifacts rescored on it — never each build's own register), and
  2. The invariant battery passes — the codified list of everything the loss cannot see: input-mass parity, tail concentration, effective sample size, band delivery, coverage/degeneracy, spine agreement (new here). Every battery entry was purchased with a specific incident; the battery is versioned, machine-checked in one batched pass, and recorded in both manifests.

Nothing else gates a ship. Adjudications and fences are receipts inside rule 2, not extra rules; the per-arm sprawl this epic retires was rule 2 metastasizing across two lineages. Loss is the objective the optimizer games — it can be rule 1 only because rule 2 checks what it can't say.

The design (ratified direction, 2026-07-29)

One base pool, one lineage, every dataset a record count:

  1. Two household spines + one detail channel. CPS-ASEC and ACS PUMS as peer household spines; PUF stays a cloned tax-detail channel on household-shaped shells (PUF records are filers — no household structure — so it cannot be a peer spine; today's puf_tax_detail channel is already this).
  2. Dense is not a category — it's k = N. The full pool with no selection, calibrated dense_no_l0.
  3. Sparse is not a category — it's an exact record count. Releases are labeled by cardinality (…-k57240, …-k250000, …), and the artifact carries exactly that many households, gate-asserted (len(support) == k). "sparse"/"dense" retire as dataset names; future sizes are just new k values.

Exact-k selection (three stages, all seams existing)

  • Stage 1 — L0 to the ballpark, keeping probabilities. The hard-concrete gate machinery (populace-calibrate/gates.py) already yields a per-record open-probability π_i; today we threshold it. Instead, stop at π.
  • Stage 2 — certainty units + fixed-size πps draw. Records with π_i ≥ π_hi are force-included (certainty units, standard survey practice); the remaining budget k − |certainty| is drawn from the boundary mass with a fixed-size inclusion-probability-proportional design (conditional Poisson / Sampford), seed pinned in the release manifest. Deterministic given (solve state, seed); no per-target knobs anywhere.
  • Stage 3 — the existing frozen-support weight refit + full gate battery. Unchanged (calibrate_l0_refit semantics). Cell-coverage gates catch any unlucky boundary draw; a k below the feasibility floor for the target surface fails loudly, never ships thin.

Why now — the measured defects this must clear (the bar, not a hope)

The buildl ACS-multispine staging (#403; 1.61M households, ASEC+ACS) measured three defects that are the acceptance criteria for increment 2:

  1. Spine-conditional agreement. Baseline SSI incidence 2.59% (ACS spine) vs 1.85% (ASEC); $10k/$20k delta $821 vs $433 per baseline recipient. NEW GATE: spine-conditional distributions of transferred inputs must agree (declared tolerance) before calibration is attempted.
  2. Effective-support collapse. buildl calibration landed ess_fraction 0.003 (≈4,854 effective of 1.6M) with persons-per-household 1.50 (ASEC) vs 2.93 (ACS). A dense arm that is effectively 5k records is worse than an honest exact-k release. Gate on ess and per-spine composition.
  3. Baseline SSI overshoot vs SSA on that surface (no SSI target there then; the v9.x register carries the SSA bands now).

The transfer machinery these gates need did not exist in June and does now: donor joint-vector fidelity (#570's construction assertion + bijection), the deterministic mass-conserving tail-transfer stratum (#568), stage-attributable concentration gating (#571), reviewed-exclusion discipline (#564/#574), and arm-fence receipts in both manifests (#577).

Increments

  1. #395 operator ordering — assemble spines first; clone/impute/derive/seed/simulate/calibrate as spine-blind operators downstream. (Existing charter; becomes increment 1 of this epic.)
  2. ACS transfer re-run under the #567-era machinery, gated on the #403 defects above (spine-agreement + ess + composition gates).
  3. Exact-k selection (stages 1–3 above) from the healthy pool; first ladder: k = N (dense), k ≈ today's 57k contract, and one smaller demo-class k. Spine-aware near-duplicate handling in stage 2 so support composition is not solver noise.
  4. Lineage unification: the acs_local overlay (#512) re-bases onto this pool (local = filter/reweight the one national dataset); per-arm adjudication registers re-adjudicate against the unified pool (SSI fences #577, QRF registers, STCG parity #574 all RE-EXAMINED, none carried forward silently).

Non-goals / doctrine carried forward

  • No per-target calibration knobs; no loss shaping; gates are never loosened to fit outputs.
  • Raw-only doctrine intact: ACS-spine records carry transferred inputs with full provenance; measured frame values untouched.
  • Selection is deterministic given the manifest (solve state + pinned seed); the notary/CI split unchanged.
  • Release ids state the exact count; nothing labeled "sparse" or "dense" going forward.

Context receipts

  • Certified national default today: 57,240 = 35,040 puf_tax_detail clones + 22,200 asec (channel crosstab of the released H5); base pool 352,932 = 184,080 + 168,852 — zero ACS units in either arm; --acs-h5 is input transfer only.
  • CD carrying capacity is the motivating example: ~130 households/district on the 57k (47.4% within-10 on the CD surface) vs 65.1% on the full pool.
  • Relates: #395 (increment 1), #403 (acceptance bar), #512 (overlay re-base), #567/#577 (dense-arm adjudications to re-examine), #445 (frozen-support discipline), #462/#568/#570/#571 (transfer machinery).

Sequencing: starts after the dense-P3 diagnostic publishes (#567 close-out). Build-Q-scale epic.


Scope hardening (Max ratified 2026-07-29 evening — this section governs)

  1. One suite per country. The separate local-area artifact is RETIRED, not re-based. populace_us_2024_acs_local (and the #512 lane) ends as a product: there is no "local file." Every release of the one pool carries the full geography ladder and geo targets; every file is local — the difference between releases is fidelity, and fidelity is published, not implied by a product name. The k-ladder's scoreboard reports within-tolerance by geography level (national / state / CD) per release, so choosing a k IS choosing a local sample size with its measured consequences in front of you.
  2. Record count is the only within-country variation. No sparse/dense, no local/national, no arm names. populace-us-2024-k57240, -k353000, -k5000 — one lineage, one register, one battery, k values.
  3. Non-US source tiers (the one extra axis, named by provenance, never by quality). Where a country has a licensed-microdata build and a public-derivable build (UK: FRS-licensed vs the public basis), the tier is named by its spine source license class: proposal populace-uk-2023-frs-k<N> (restricted distribution) and populace-uk-2023-public-k<N>. Rules: the tier token names what the spines are built from, never adjectives ("true", "full"); both tiers run the same register and battery; the public tier's scoreboard shows its fidelity gap to the frs tier as a published number. Tier token per country ratified with that country's onboarding. US ships tierless (single public-safe basis).
  4. UK is in scope for the same end state: one UK pool, exact-k ladder, OA-ladder geography on every file, both tiers, same two-line acceptance rule. The UK lineage (June 19 vintage) gets the O/P-era battery and doctrine as part of this epic, not as a separate arc.

Increment 4 is amended accordingly: not "overlay re-base" but overlay retirement — the acs_local consumer surface (policyengine.py named access) migrates to a k-release of the one pool, then the artifact family is closed out with a deprecation note in the bundle manifest.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with populace-calibrate/gates.py and the existing calibrate_l0_refit path, then inspect the policyengine.py access named in the scope-hardening section. Trace the machinery referenced in issues #395, #403, and #567–#577 before changing the release flow. Done means the unified exact-k ladder, versioned invariant battery, manifests, and retired local artifact satisfy the stated acceptance rule.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.