PolicyEngine / PolicyEngine/policyengine-uk

Structural-trigger parameters are sampled at a fixed instant, so reforms gating structural changes can silently fail to activate

Open
#1,800 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
50
Forks
33
Avg merge
20h 58m
Merged PRs (30d)
14

Description

Summary

Simulation.__init__ creates structural reforms from parameter values once, at a fixed instant:

structural_reform = create_structural_reforms_from_parameters(
    self.tax_benefit_system.parameters,
    period_(self.default_input_period),   # 2025 in current releases
)

Because the trigger parameters (gov.contrib.cps.marriage_tax_reforms.*, gov.contrib.conservatives.cb_hitc_household, gov.contrib.policyengine.disable_simulated_benefits, gov.contrib.scotland.scottish_child_payment) are read only at default_input_period, a reform that sets a trigger for any period not covering that instant changes the parameter but never activates the structural reform it gates. The simulation then returns baseline numbers with no error or warning.

Reproduction (policyengine-uk 2.88.18)

Married couple, one earner at £80,000 (fails the marriage allowance income condition under current law):

  • Reform {"gov.contrib.cps.marriage_tax_reforms.expanded_ma.remove_income_condition": {"2025-01-01.2030-12-31": true}} → structural reform activates, marriage_allowance for 2026 becomes £1,260.
  • Same reform keyed {"2026-01-01.2026-12-31": true} → parameter verifiably changes on the simulation's own system, but marriage_allowance for a 2026 calculation stays £0.

A second, compounding issue

Dict reforms passed as Simulation(reform={...}) go through Scenario.from_reform, which returns a scenario with applied_before_data_load=False — so the parameter changes are applied after create_structural_reforms_from_parameters has already run. On that path a structural trigger can never activate, regardless of period. Constructing the Scenario manually and setting applied_before_data_load=True restores correct behavior; should from_reform default to early application for parameter dicts?

Same pattern in policyengine-us

policyengine_us/system.py does the equivalent with DEFAULT_START_DATE = str(CURRENT_YEAR) + "-01-01" where CURRENT_YEAR = 2024 is hardcoded — the sampling instant there is both fixed and increasingly stale. A fix here probably wants mirroring there.

Suggestions

  • Sample trigger parameters across the periods the reform actually touches (or re-evaluate per requested calculation period), or
  • at minimum, document the constraint that trigger reforms must cover the sampling instant, and
  • consider Scenario.from_reform defaulting to applied_before_data_load=True for dict reforms.

Context

Found while fixing UK /calculate in the PolicyEngine household API, which now applies reforms via a manually-constructed Scenario with applied_before_data_load=True and documents the sampling constraint in its tests.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Simulation.init, create_structural_reforms_from_parameters, and Scenario.from_reform to trace when trigger parameters are sampled and when reforms are applied. Compare the equivalent logic in policyengine_us/system.py, then inspect the tests mentioned in the issue. Done means period-specific structural triggers behave correctly, or the sampling constraint is explicitly documented and tested.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.