PolicyEngine / PolicyEngine/microcosm
Release reform-validation tail derives UNKNOWN county for records with valid county_fips (blocks manifest)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 4
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 94
Description
Reform-validation tail derives county = UNKNOWN for records whose exported artifact carries valid county_fips
The release tool's reform-validation tail (tools/build_us_fiscal_refresh_release.py, the batched zero-variable-reform simulation built from the frame) crashes at the 2025 period with ParameterNotFoundError: 'gov.local.md.flat_rate.UNKNOWN' — even though the exported artifact's MD households all carry valid county_fips. This blocks release_manifest.json (rc=1) after all certification gates + the export-mass gate + the H5 have already been written.
Root cause of the raise itself is a policyengine-us fragility (md_withheld_income_tax has no UNKNOWN-county guard — see cross-linked issue). But the reason an UNKNOWN county appears at all is harness-side: the exported dataset is geographically clean, so the batched reform-validation construction is losing/mis-deriving county for some MD record.
Three-way clean-artifact evidence (Build I sparse, rmloss100)
- Exported H5 via the tail's own loader (
USSingleYearDataset(file_path=…)→Microsimulation,_write_reform_validationline ~4691): all 1,091 MD households map to valid MD counties at both 2024 and 2025; ZEROUNKNOWNcounties anywhere in the 57,240 households;md_withheld_income_tax@2025computes cleanly ($28.13B). The H5 persists validcounty_fipsfor all 24 MD jurisdictions (24001…24510). - Frame path is clean: the target-frame checkpoint carries
county_fips/state_fips/place_fips;_dataset_from_framepreserves them; the full-frame reform-validation dataset also computesmd_withheldwith 0 UNKNOWN. - Full-frame reforms are clean: neutralizing
salt_deduction,interest_deduction, andqualified_business_income_deductionand computingincome_taxon the full frame all succeed. The crash appears only in the batched construction (_reform_household_income_tax/_batched_reform_validation_simulate_factory_from_frame, ~lines 2357–2431:_select_households_by_position(base_frame, positions)→_dataset_from_frame(batch_frame)→Microsimulation(dataset=reformed_dataset, reform=reform)at 2025).
So a per-batch frame slice appears to drop or fail to carry county_fips (or its downstream county) for at least one MD household, which then trips the unguarded MD flat-rate lookup.
The live default is more exposed, not less
The current live/default artifact (c2065b64) persists no county_fips at all (only state_fips), so its reform-validation would derive UNKNOWN for every MD household and hit this row immediately if the tail reached it. Build I actually persists correct county geography; earlier runs' tails were interrupted before reaching this row, which is why it had not surfaced.
Traceback (verbatim)
File ".../salt_deduction.py", line 16, in formula
File ".../state_and_local_sales_or_income_tax.py", line 13, in formula
File ".../gov/states/md/tax/income/md_withheld_income_tax.py", line 28, in formula
flat_rate_withheld = p_local.flat_rate[safe_county] * reduced_agi
...
ParameterNotFoundError: The parameter 'gov.local.md.flat_rate.UNKNOWN' was not found in the 2025-01-01 tax and benefit system.
Workaround (used for Build I)
--skip-reform-validation completes the release and writes release_manifest.json; the reform-validation report is optional and all certification output (gates, export-mass parity, H5) is produced before the tail. The dataset is unaffected.
Suggested fix
Ensure the batched reform-validation frame slices carry county_fips/county end-to-end (so the reformed dataset matches the exported artifact's geography), and/or adopt the policyengine-us UNKNOWN-county guard (cross-linked issue) so the validation report is robust to any dataset lacking county detail (e.g. the current PUF-derived default).
Cross-reference (root-cause formula fragility): https://github.com/PolicyEngine/policyengine-us/issues/8975
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in tools/build_us_fiscal_refresh_release.py around _reform_household_income_tax and _batched_reform_validation_simulate_factory_from_frame, then inspect _select_households_by_position and _dataset_from_frame. Run the reform-validation tail on the clean Build I frame and compare county_fips/county through each batch slice. Done means the batched validation preserves valid Maryland county geography and completes without an UNKNOWN county lookup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- release
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100