PolicyEngine / PolicyEngine/uc-rebalancing

HE-only leg decomposition is structurally zero; SA uplift labels show annual rate as cumulative

Open
#3 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Two bugs in the leg-decomposition pipeline, found by tracing how policyengine_uk actually applies the rebalancing reform. Filing to confirm with someone closer to the data before fixing.

Bug 1: reform_he_only is identical to counterfactual

src/uc_rebalancing/pipeline.py:96-117 describes the four sims with this mental model:

"the standard allowance uplift is gated by a parameter formula, while the health element fix is baked into the uc_LCWRA_element input column of the enhanced FRS dataset"

Neither half is accurate. In policyengine_uk/scenarios/uc_reform.py, both legs are applied by the same add_universal_credit_reform simulation modifier via set_input, and both are gated by rebalancing.active(year) (lines 21 and 39 of uc_reform.py). Neither uc_standard_allowance nor uc_LCWRA_element has a parameter formula that branches on rebalancing.active — they read CPI-uprated standard_allowance.amount / elements.disabled.amount respectively.

policyengine_uk/simulation.py:114-116 calls the modifier unconditionally at the end of every Microsimulation.__init__, after parameter changes from reform= are applied. So passing reform={"gov.dwp.universal_credit.rebalancing.active": {...: False}} makes the modifier hit continue and skip both legs.

Net effect on the four sims:

Sim active LCWRA set? SA set? LCWRA stripped? Effective
counterfactual False no (skipped) no (skipped) no-op both off ✓
reform True yes yes no both on ✓
reform_sa_only True yes → stripped yes yes SA on, HE off ✓
reform_he_only False no (skipped) no (skipped) no both off ✗

So reform_he_only ≡ counterfactual, and every HE-leg field on the dashboard (summary_he, by_decile_he, winners_losers_he) is computed as reform_he_only − counterfactual ≈ 0. The HE column is structurally zero, not a small residual.

reform − counterfactual (total) and reform_sa_only − counterfactual (SA leg) are both correct.

Fix options
  • Cleanest: define a local Scenario that runs only the HE half of add_universal_credit_reform. Build reform_he_only with reform={active: False} (so the bundled modifier skips) and attach the HE-only scenario via the scenario= kwarg.
  • Mirror the existing trick: leave active=True, then after construction strip uc_standard_allowance for the schedule years via holder.delete_arrays(period=year) — same pattern as _strip_rebalanced_lcwra but on the SA variable. Reverts SA to formula while keeping the modifier-set HE values.

The per-claimant grid HE path (pipeline.py:394-399) uses set_input directly on the merged situation, so the grid HE column is unaffected by this bug.

Bug 2: SA uplift labelled as cumulative, but it's annual

gov.dwp.universal_credit.rebalancing.standard_allowance_uplift:

2026-01-01: 0.023
2027-01-01: 0.031
2028-01-01: 0.040
2029-01-01: 0.048

The modifier compounds these year-over-year: previous_value * (1 + uplift(year)). So cumulative 2025→2029 is 1.023 × 1.031 × 1.040 × 1.048 ≈ 1.150 (~15%).

Sanity check against DWP IA's £725/yr single-25+ uplift by 2029/30: SINGLE_OLD 2025 SA = £400.14/mo = £4,801.68/yr; compound to £5,522/yr → +£721/yr. Matches. The parameter values are annual rates.

But src/uc_rebalancing/scenarios.py labels them as cumulative:

  • scenario_label formats FY 2029/30 as "4.8%"
  • policy_description says "reaching {final_pct:.1f}% cumulatively by {final_year}" → "reaching 4.8% cumulatively by 2029"

Both should reflect either the annual rate per year ("2.3% for 2026/27, ... 4.8% for 2029/30") or the cumulative figure recomputed from compounding (~15.0% by 2029/30).

Asks

  1. Confirm the read of add_universal_credit_reform — are both legs really co-gated by active, or is there a separate input-column path I'm missing in the bundled enhanced FRS dataset that re-supplies uc_LCWRA_element for reform_he_only?
  2. Confirm whether standard_allowance_uplift parameter values are meant to be annual rates (as the modifier and DWP £725 figure suggest) or cumulative (as the dashboard currently labels them).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing src/uc_rebalancing/pipeline.py:96-117 and 394-399 alongside policyengine_uk/scenarios/uc_reform.py and policyengine_uk/simulation.py:114-116. Verify the four simulation paths and the standard_allowance_uplift interpretation against the scenario labels in src/uc_rebalancing/scenarios.py. Done means the HE-only simulation is distinct from the counterfactual and the SA labels accurately describe annual or compounded rates.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.