PolicyEngine / PolicyEngine/microcosm
Export input-mass-parity gate references the raw base, flagging correct calibration gains on PUF-imputed columns
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 4
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 94
Description
Summary
_export_input_mass_gate (in tools/build_us_fiscal_refresh_release.py) compares the calibrated export frame against the raw pre-calibration base frame with a symmetric ±50% tolerance. For a dense parent built from a raw pooled-ASEC base, this fires on exactly the PUF-imputed income columns that calibration is supposed to scale up toward SOI/CBO fiscal targets — flagging correct, target-aligned masses as failures. The gate as wired is mis-referenced for the dense-parent-from-raw-base case. It should either (a) reference a certified release instead of the raw base, or (b) thread a reviewed-exclusions register for the PUF-imputed columns, or both. It should not be silenced globally with --allow-input-mass-drift (that would also mask the genuine #278 zeroing it was built to catch).
Where it bit
Build F (populace#299), attempt 5, dense-default run on the certified 3-year ASEC pool (337,704 hh). Calibration was excellent — final_loss 0.0416, within-10% 86.1% (beating certified f0af251's 0.0423 / 86.4%) — and every structural gate passed (target_profile_coverage, degenerate_input_signal, base_population_scale, immigration_composition, ecps_parity, validation_input_coverage). The only failure was _export_input_mass_gate, which raises after the export frame exists (which is why earlier attempts, dying at the CHIP gate, never reached it).
14 columns flagged, all export >> base (gains, not losses):
| column | export vs base | drift |
|---|---|---|
| long_term_capital_gains_before_response | 7.02e11 / 2.12e11 | +230.7% |
| short_term_capital_gains | 9.58e10 / 2.90e10 | +230.7% |
| miscellaneous_income | 1.76e10 / -8.15e9 | +315.6% (base is negative → % meaningless) |
| partnership_self_employment_net_earnings | 3.47e10 / 1.25e10 | +177.2% |
| qualified_dividend_income | 3.20e11 / 1.32e11 | +142.3% |
| non_sch_d_capital_gains | 7.43e10 / 3.38e10 | +119.7% |
| tax_exempt_interest_income | 6.13e10 / 2.82e10 | +116.9% |
| charitable_cash_donations | 2.61e11 / 1.38e11 | +88.9% |
| charitable_non_cash_donations | 5.57e10 / 3.05e10 | +82.6% |
| rental_income | 3.84e11 / 2.13e11 | +80.4% |
| non_qualified_dividend_income | 1.35e11 / 8.22e10 | +64.3% |
| unemployment_compensation | 3.35e10 / 2.05e10 | +63.3% |
| home_mortgage_interest | 5.31e11 / 3.45e11 | +54.1% |
| first_home_mortgage_interest | 5.05e11 / 3.33e11 | +51.9% |
(tolerance 0.5, minimum_reference_total $1B, 41 columns checked, reviewed_exclusions empty.)
Why these masses are RIGHT (not a calibration bug)
Compared each flagged column's calibrated national mass to its SOI/CBO national target (the honest reference, vs the un-aligned raw base):
qualified_dividend_income: export $320B; CBO qualified-dividend target $354B → est is hitting the target (−9.7%). The gate flags +142% vs raw base for a mass that is slightly under its fiscal target.- capital-gains family (LT/ST/non-Sch-D): CBO
net_capital_gaintarget $1.291T, calibrated est $0.798T → −38.2%, i.e. even after +230% vs raw base, capital gains still undershoot CBO. The increase the gate flags is genuinely needed and still insufficient. partnership_self_employment_net_earnings→ CBOnet_business_incometarget $1.916T, est $1.917T → +0.1%, essentially exact.- AGI control: −0.9%.
CPS structurally under-reports capital, dividend, and business income; PUF imputation seeds it low; calibration reweights toward SOI/CBO. The ±50%-vs-raw-base band asks calibration not to do its job. miscellaneous_income additionally has a negative base total, making a relative-drift check ill-defined.
Why the raw-base reference is the wrong yardstick here
The gate's docstring states its purpose: "L0 selection and reweighting optimize the target surface; an untargeted input column can lose its mass without moving any residual. The export must keep every material input base." It is the #278 mass-LOSS guard (sparse selection zeroing IRA/HSA/pension/childcare — candidate << reference). It applies abs(drift) > tol symmetrically, so it also fires on calibration gains — the opposite of the defect it targets.
Provenance corroborates: input_mass_parity_gate was added in #279 (44868be), after the certified f0af251 (Jun 20) and the live-default lineage. Neither certified release manifest records input_mass_parity (only the base_population_scale person-population mass_repair, which build-f also passes). The certified 57k live default was built without this gate and would itself fail export-vs-raw-base on the same PUF columns. There is no pre-calibration income-mass alignment step in the certified pipeline.
Proposed fix
- Reference a certified release, not the raw base, for the export-vs-reference comparison (the underlying
input_mass_parity_gatealready supportsreference_totalsfrom a reference H5; the release builder already has an--input-mass-reference-h5path used by_input_mass_reference_gate). Against a certified reference, calibration-driven upward alignment of PUF income is in-band, while a genuine sparse zeroing still fails. - And/or thread
reviewed_exclusionsthrough_export_input_mass_gate(the underlying fn takes it; the wrapper currently doesn't) for the PUF-imputed columns, each with the SOI/CBO-target justification above, so the register documents why these columns legitimately move. - Keep the loss/zeroing arm strict (candidate == 0 or candidate << reference must still fail — that is the #278 signature). Consider making the gain-side tolerance reference the fiscal target rather than the base, or asymmetric.
- Do not rely on
--allow-input-mass-driftas the standing answer — it disables the whole gate, re-opening #278.
Context
Found during the Build F certified-lineage rerun (populace#299). This is the last gate the dense parent trips; with it correctly referenced, the dense build passes on the certified pool.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in tools/build_us_fiscal_refresh_release.py at _export_input_mass_gate, then trace the underlying input_mass_parity_gate and the existing --input-mass-reference-h5 path used by _input_mass_reference_gate. Re-run the dense Build F certified-lineage case; done means legitimate PUF-imputed gains no longer fail while the #278-style mass-loss or zeroing case remains guarded.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100