PolicyEngine / PolicyEngine/microcosm

Build E: certification-candidate staging build from post-#287 main (fix target_compilation OOM first)

Open
#299 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Goal

Run Build E — the certification-candidate US staging build — from current main, which now carries the complete recipe: #266 immigration, #279 input-carry, #280+#287 full-surface chained target aging (default ON, 1,480 targets, ×1.1203 chained TY2022→TY2024; national AGI $15.29T→$16.62T, income tax $2.11T→$2.36T at the target level), #286 stuck-at-default input gate, #283 L2/concentration diagnostics, pinned Ledger bundle enforcement.

Two prerequisites (both builds B and D died before calibration)

  1. Fix the target_compilation kill. Builds B (us-2024-immigration-266-b-20260702T114815Z) and D (populace-us-2024-32fcf6b-de00cae57dc7-20260702T145207Z, nonpreemptible, 220 GiB) both died at the same locus: mid-target_compilation on the 224,026-household pool, with 2 of 5 JCT reform materializations complete — D's death was a hard kill, most likely OOM (the driver's own warning puts dense worst-case near ~120 GB; observed exceeds 220 GiB), with the 10h timeout secondary. The JCT reform materialization step (full engine runs over the pool, despite #236's streaming) is the prime suspect. Options: chunk/stream the reform materialization further, checkpoint per-reform (the #227 checkpoint machinery may already support resume — verify why the restart lost the materialization cache in B), or provision >300 GiB.
  2. Inputs per #287's provenance contract: the pinned, hash-verified Ledger bundle (f6fa6152…) — do not substitute unpinned facts. Confirm the base pool H5 (populace-us-2024-cd-concept-budget-dbbdcec-512e-b2500-r2-20260627T022640Z, 224,026 hh) carries the #278 input columns so the #286 gate passes with #279's carry (if the base itself lacks them, the base needs a rebuild first).

Guardrails (unchanged)

Staging only (policyengine/populace-us-staging); never touch policyengine/populace-us, its latest.json, or root files; certification is a separate explicit human decision after the results comparison vs the published baseline (2025: net income $14.50T, tax $3.43T, AGI $16.01T, income_tax $2.15T, poverty 13.7%, citizens 100%, IRA/HSA/childcare inputs absent, OBBBA CTC-SSN Δ −$37.1B vs JCT).

Context: failure analysis on #204 (2026-07-03 comment); Modal runner state in the B/D orchestrator's STATE.md (session scratchpad).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the failure analysis on #204 and the B/D orchestrator STATE.md, then inspect the target_compilation step and the #227 checkpoint and #236 streaming context. Done means the target compilation completes without the observed memory failure, the pinned Ledger bundle and #278 input columns pass the #286 gate, and Build E runs in policyengine/populace-us-staging.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, data-engineering
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.