PolicyEngine / PolicyEngine/microcosm-benchmarks
Is the populace-us ↔ current-eCPS parity drift blocking downstream (CRFB taxation-of-benefits, PolicyBench)?
Nobody has claimed this yet.
- Dominant language
- No language data
- Stars
- 0
- Forks
- 0
- Avg merge
- 29m
- Merged PRs (30d)
- 1
Description
Question
Is the parity drift between the certified populace-us default and the current enhanced CPS blocking for downstream consumers — specifically the CRFB taxation-of-benefits work and PolicyBench? This issue exists to collect that impact assessment. The fix for the drift mechanism is tracked separately in PolicyEngine/populace-benchmarks#1; this issue is only about whether the current state blocks downstream use while that lands.
The facts (reproduced, not inferred)
Running the build's own parity checker against the sha-verified certified artifact (populace_us_2024.h5, sha256 f32c2e5e…, == the certified default in policyengine.py bundle 4.16.2):
- vs the eCPS the build was gated against → 0 gaps (the recorded "parity 0" is real and reproduces)
- vs today's eCPS → 12 gaps
It is reference drift, not a gate escape: eCPS has since populated layers the certified populace zeroes, and the parity reference was never pinned. exported_nonzero still passes (populace's own columns are clean).
What diverges (current eCPS populates, certified populace = 0)
Federal tax inputs that feed net tax (no formula recovery — genuinely 0 in populace):
| variable | share of records eCPS populates |
|---|---|
amt_foreign_tax_credit |
9.3% |
general_business_credit |
5.4% |
excess_withheld_payroll_tax |
4.6% |
prior_year_minimum_tax_credit |
4.0% |
early_withdrawal_penalty |
3.3% |
other_credits |
2.8% |
self_employed_health_insurance_ald |
7.4% (formula-owned; 0 because upstream SE-health inputs absent) |
Reported / comparison aggregates (not formula-consumed; used in reported-vs-modeled analysis):
spm_unit_net_income_reported (99.4%), spm_unit_total_income_reported (97.7%), ssi_reported (1.5%).
Not affected: retirement contributions (roth_ira/roth_401k/traditional_401k/SE-pension) are exported as _desired inputs and recovered by the engine formula (~15–17% nonzero in simulation) — not gaps.
Likely impact surface
For the ~3–9% of records each credit touches, populace computes lower-magnitude federal tax than current eCPS (those credits/penalties are simply absent). Distributional and revenue figures for affected slices will differ from an eCPS-based run. The reported aggregates only matter to analyses that read the reported (not modeled) columns.
For the CRFB taxation-of-benefits agent
The TOB v2 work runs on the populace base. Question: do any of the above — the six federal credits/penalties, the SE-health ALD, or the spm_unit_*_reported aggregates — enter the taxation-of-benefits analysis path or its targets/validation? If TOB is driven by Social Security taxability, AGI thresholds, and the standard income variables (which are unaffected), this may be immaterial; if it nets against these credits or compares to reported SPM income, it may not be. Please assess whether this blocks the TOB numbers or is tolerable until PolicyEngine/populace-benchmarks#1 closes the data gaps.
For the PolicyBench agent
Does PolicyBench score anything that depends on these layers — i.e., do any benchmark cases exercise the six credits/penalties or compare against eCPS on the reported aggregates? If the benchmark suite would shift (or if it pins eCPS as a reference that populace is now expected to match), quantify the score delta and state whether it blocks.
Status of the fix (context, not the question here)
PolicyEngine/populace-benchmarks#1 is closing the mechanism: a reference-pinned, recorded, reconstructable parity runner (branch pin-parity-reference, in review) so "parity 0" is reproducible against an explicit eCPS revision, plus a CI drift-check. The data gaps (importing the six PUF-derived credits; populating SE-health upstream) are the open work that this blocking assessment should prioritize or de-prioritize.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the build's parity checker against the certified populace_us_2024.h5 artifact and reviewing the certified default in policyengine.py bundle 4.16.2. Then inspect the CRFB taxation-of-benefits path and PolicyBench cases for dependencies on the listed credits, SE-health ALD, or reported aggregates. Done means documenting whether each path is affected, quantifying any benchmark shift, and stating whether downstream work is blocked.
Written by the indexing model from the issue text.
Assessment
- Domain
- analytics, data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100