PolicyEngine / PolicyEngine/microcosm
base-m collapses partnership_self_employment_net_earnings to signed near-cancellation (+47.7B/−48.1B legs)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 4
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 94
Description
Found by the export-mass parity gate on Build M sparse attempt 10 (release id populace-us-2024-buildm-sparse-rmloss100-5482d38-20260715T114657Z): partnership_self_employment_net_earnings exported −$22.72B against the reference's +$61.74B (−136.8%, far beyond the ±50% band).
The gate is right that mass was lost, and the loss is in the base pool, not the solve:
| artifact | weighted national mass |
|---|---|
| live-default reference c2065b64 (us-data era, PUF-direct values) | +$61.74B |
| base-j pool (865,046 persons, base weights) | +$12.21B |
| base-m pool (same persons, rebuilt stages) | −$0.35B |
| Build M sparse export (57,240 hh, solved weights) | −$22.72B |
base-m's column decomposes to a positive leg of +$47.71B against a negative leg of −$48.06B — near-perfect cancellation. The column is signed, sparse (4,467 nonzero of 865k persons, ~0.5%), and heavy-tailed on both sides: the worst case for leaf-averaged QRF imputation. base-j had already compressed it (+$61.7B → +$12.2B); the base-m rebuild (staged builder #406 chains, #415 leaf bound in effect) pushed the net to zero. The solve then amplifies in either direction because nothing pins the column — Build J's solve happened to drag it up in-band (−37.6%), Build M's dragged it down and out.
Reachability is NOT the issue (unlike the miscellaneous_income / #393 case): at ratio-5 the positive leg alone could reach $238B. The pool's signed near-cancellation is.
Remedy, in order:
- Diagnose the QRF chain for this column in the staged base builder: does the near-discrete guard, the leaf bound, or donor-pool construction destroy the positive/negative separation? Compare per-leg distributions donor-PUF vs imputed.
- Fix and rebuild base (per-target checkpointing makes this a single-chain rebuild, not a full base rebuild).
- Longer term (architecture week): identify the column with a real target the way Build H did for
estate_income— SE-earnings concepts can anchor to SOI Schedule SE statistics — so the solve stops treating it as a free dimension. Same applies to every unpinned column the parity gate watches.
A reviewed parity exclusion for this column ships with Build M (register entry cites this issue); the exclusion lifts with the rebuild, exactly like the miscellaneous_income/#393 pattern.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the staged base builder and its QRF chain for partnership_self_employment_net_earnings, using the export-mass parity gate and the base-j/base-m comparison as context. Compare donor-PUF and imputed positive/negative leg distributions, then use per-target checkpointing to rebuild the affected chain. Done means the signed base mass is no longer near cancellation and the reviewed parity exclusion can be lifted.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100