PolicyEngine / PolicyEngine/microcosm
Dense arm fails input-mass parity on miscellaneous_income (−58%); Build M dense needs a target-side remedy
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 4
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 94
Description
Summary
The Build J-vintage dense arm (337,704 households, full pool, no-L0) fails exactly one hard gate: input mass parity on miscellaneous_income at −58.0% (export frame $19.92B vs reference $47.40B; band ±50%, $1B floor). All 34 other export columns pass; the release tooling correctly refused to emit release_manifest.json/H5. Build M's dense re-run must carry a remedy — this issue is the runbook entry.
Evidence (staging run 2026-07-10, release id populace-us-2024-buildj-dense-warmstart-c10f7bb-20260710T231332Z)
- Calibration quality is NOT the problem: final_loss 0.039657 (beats Build H dense 0.04018), within-10% 87.03%, ESS 65,959, coverage gate 70/70 PASS including the restored SCF asset and SNAP work-requirement columns.
- The failure is a robust property of the optimum, not a convergence artifact: cold direct solve −54.1% → 1500 warm-start epochs improve total loss but push misc further out to −58.0%.
- Dense-vs-sparse inversion vs Build H: Build J sparse holds misc +11.46% (in band) while dense fails; Build H was the reverse. The Build-J-specific driver is the newly restored input columns adding competing structure the full-pool solve satisfies at misc's expense — misc's SOI Table 1.4 target ("other income less loss", ~$52.84B aged) is sign-unstable and low-weight in the 5,534-spec surface.
Non-remedies (ruled out)
- Reviewed exclusion: not eligible — the reference mass ($47.4B) sits near the SOI target ($52.84B), so the band is achievable in principle.
--allow-input-mass-drift: gate bypasses are exactly what the campaign exists to eliminate.
Candidate remedies for Build M dense
- Raise the loss weight on the miscellaneous_income target family so the full-pool solve cannot trade it away (calibration-config change; measure the cost to other targets).
- Band-aware penalty: add the export-mass band itself as a soft constraint for sign-unstable SOI net concepts.
- If neither holds the band, split the net target into gross/loss legs so the solver stops exploiting sign cancellation.
Full run report (stage timings, RSS profile, shas) is preserved in the staging release dir alongside input_mass_parity.json.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the preserved staging release directory and inspect input_mass_parity.json and the full run report for the Build J dense result. Compare the dense and sparse outcomes, then evaluate the candidate calibration remedies against their effects on other targets. Done means Build M dense passes the miscellaneous_income parity band without a gate bypass and can emit the release manifest.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100