PolicyEngine / PolicyEngine/policyengine-us
Atomic inputs: one fact, one input — eliminate arbitration between redundant input variables
- Dominant language
- Python
- Stars
- 162
- Forks
- 212
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 97
Description
## The rule
**One fact, one input, at its natural entity.** Derived aggregates are formulas, never alternative inputs; the model never arbitrates between two inputs describing the same real-world fact. This extends #8162's "inputs are pure leaves" (no formulas/adds/guards on inputs) to *unique* leaves: redundancy between pure leaves is where silent double-counts, ignored inputs, and fallback bugs breed — #9004/#9142 (mortgage) being the type specimen: two input vocabularies for one fact required arbitration logic, and the arbitration itself was the bug.
"Natural entity" is the entity where the fact actually attaches — usually person for money amounts, but e.g. loan-level facts (balance, origination year) park at tax_unit for want of a loan entity. Atomic ≠ everything person-level.
## Inventory (verified on main, 2026-08-13)
| Case | Inputs in conflict | Arbitration site |
|---|---|---|
| **Mortgage interest** | person `home_mortgage_interest` vs tax-unit `first/second_home_mortgage_interest` | `mortgage_interest_structure.py` (`where(structured > 0, …)`) and again in `home_mortgage_interest_share` |
| **Heating expenses** | person `heating_expense_person` vs spm-unit `heating_cooling_expense` vs per-fuel spm-unit `pre_subsidy_electricity_expense`/`gas_expense`/`fuel_oil_expense` | per-program: `dc_liheap_payment.py`, `il_liheap_base_payment.py` (each arbitrates differently) |
| **Subset markers** | `military_retirement_pay` must *also* be included in `taxable_pension_income` by the user; no arbitration, just a convention — forget either side and results are silently wrong (documented in the VT Act 71 test comments) | none (that's the problem) |
| **`_reported` overrides** | `ssi_reported`+`use_reported_ssi`, `medicare_part_b_premiums_reported`, `spm_unit_net_income_reported`, two state subtraction `_reported`s | explicit toggles |
Proposed disposition: migrate the first two to single inputs; add a validation check for subset markers (subset ≤ total); **exempt the `_reported` class explicitly** — data-vs-simulation switches are a deliberate, named pattern, not redundancy.
## Mortgage migration (staged — single-shot removal is unsafe)
Load-bearing constraint, verified in policyengine-core: the dataset loader **silently skips** h5 columns whose variable no longer exists (`simulation.py`: `if variable_name not in self.tax_benefit_system.variables: continue`). Certified microcosm h5s carry the structured columns, so deleting the inputs today would silently zero mortgage interest for existing data. (Core-side warning proposed separately.)
Also verified in microcosm: the structured interest columns are *derived* in the build's `mortgage_conversion` stage by splitting the person-level `home_mortgage_interest` total — the model then sums them back. The split is pure round-trip redundancy even inside the data pipeline; only the balances/origination years (needed for the §163(h)(3)(F) two-vintage caps, and never person-attributable — loans are joint) carry independent information.
1. **Now (pe-us):** flip arbitration — person-level `home_mortgage_interest` is canonical; structured interest is a deprecated fallback used only when the person-level sum is zero. Structured interest inputs documented as deprecated pointing here. Microsim-neutral whenever the h5's person column and split sum agree (they come from the same stage total; microcosm to confirm per-record reconciliation on the current certified build — receipt requested in the companion issue).
2. **Microcosm:** stop exporting `first/second_home_mortgage_interest` (keep person interest + balances + origination years) in the next certified build.
3. **After the first certified release without the columns (pe-us):** delete the structured interest inputs and the fallback (`removed` fragment). The known first/second interest split has no model consumer — the deduction formula only ever uses the sum.
Known line-item to resolve at stage 3: the tax-unit total counts all members while `home_mortgage_interest_share` allocates only across head/spouse — pick one convention (head/spouse is the defensible line for a filers' deduction).
## LIHEAP follow-up (separate PR)
Pick one heating-expense vocabulary (person-level `heating_expense_person` is the atomic candidate), migrate DC/IL/MA formulas, deprecate the rest on the same staged pattern (they are also microdata-supplied).
Refs: #9004, #9142, #8162, populace/microcosm#511.
Contributor guide
Research direction
Start with mortgage_interest_structure.py and the DC/IL LIHEAP formulas, then inspect simulation.py's handling of missing variables and the certified microcosm H5 inputs. Trace the staged mortgage migration and verify per-record reconciliation before changing exports or inputs. Done means redundant vocabularies have a documented migration path, subset markers are validated, and _reported switches remain explicitly exempt.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, data
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100