PolicyEngine / PolicyEngine/microcosm

ACS-multispine staging: SSI probe finds spine-inconsistent incidence (2.6% vs 1.9% baseline rate) and weight-composition skew (ASEC 1.5 persons/hh, ESS 0.3%)

Open
#403 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Context

SSI resource-limit probe of the buildl ACS-multispine staging artifact (_buildl-runtime/acs_2024_1yr_calibrated.h5, local/unpublished: 3,583,746 persons / 1,606,726 households, ASEC+ACS spines, 742-target option_b_soi_totals calibration, simulation_ready=true). Method: exact 40k-household chunking through pe-us 1.764.6 Microsimulation (41 chunks), applying the lane's own engine-pass contract — reviewed_null_fills register + held_back_columns projection. TY2026, SSI-only metrics. Same harness as the #356/#374 records.

Headline (consistent with the certified-artifact family)

scenario multispine staging Build J certified (4.20.4) dense f0af251 anchors
baseline SSI 8.31M / $71.8B 6.71M / $57.1B 8.05M / $63.6B SSA ≈7.4M / $61B
$10k/$20k +$5.96B / +845k +$5.69B / +797k +$1.60B / +499k OACT ≈$1.0B / CBPP 224k
remove test +$16.62B / +2.41M +$18.19B / +2.45M +$16.05B / +3.23M ECPS +$15.75B

The $10k/$20k delta sits in the SCF-only family, as expected — the ACS spine's asset columns come from the same SCF transfer — so #374's blend/refit applies to this lineage too. Full removal stays in the robust +$16–18B band. Baseline SSI overshoots SSA (this surface carries no SSI target; the incumbent surfaces sat closer).

QA findings (pre-launch signals)

  1. Spine-inconsistent SSI incidence. Baseline SSI rate: 2.59% on the ACS spine (6.08M of 234.7M person-weight) vs 1.85% on the ASEC spine (2.23M of 120.3M). Ten-twenty delta per baseline recipient: ACS $821 vs ASEC $433 — the same reform scores ~2× as intensely on ACS-spine records. Points at transfer-stage distribution mismatch (assets/disability/unit structure) between the spines; relates to the #395 operator-ordering question.
  2. Calibrated-weight composition skew. Implied persons per weighted household: ASEC spine 1.50 vs ACS 2.93 (each spine holds 80.08M household-weight; person-weight splits 120.3M vs 234.7M; total 355.1M vs census ≈346M). With effective_sample_size = 4,853.9 (ess_fraction 0.003), the solve is concentrating weight on few, small ASEC households — the runbook's decision-5 l2-sensitivity trigger looks warranted before this surface ships.
  3. Consumer footnote on reviewed_null_fills.json: enum-kind entries log fill_value as a repr ("'CITIZEN'"); a consumer that applies the log verbatim writes quoted strings that the engine's enum encoder rejects. Logging raw values (or documenting the repr convention) would prevent that.

Repro scripts: ssi-asset-limits .local-results/populace-rerun-scripts/pass_h_*.py; outputs in populace-buildl-acs-multispine-* incl. by_spine.csv. Relates #395, #374, #368/#369.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the SSI repro scripts in .local-results/populace-rerun-scripts/pass_h_*.py and compare the generated by_spine.csv outputs for the ACS and ASEC spines. Check the reviewed_null_fills.json enum entries and the runbook's decision-5 l2-sensitivity trigger. Done means the spine mismatch, weight skew, and enum fill_value behavior are explained and the appropriate follow-up is recorded before launch.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.