PolicyEngine / PolicyEngine/policyengine-uk-data

Rail, bus and fuel calibrations run after base-year materialisation, so the shipped file is not uprate-invertible

Open
#479 1 comment 0 reactions 1 assignee View on GitHub

@juaristi22 is already working on this.

Since Sep 3, 2026.

Dominant language
Python
Stars
1
Forks
2
Avg merge
1d 14h
Merged PRs (30d)
4

Description

_materialize_base_year_dataset is not the final transformation before the dataset is saved. In create_datasets.py:

304-311  _materialize_base_year_dataset  ->  uprate_dataset(frs_calibrated, base_year)
319      calibrate_rail_subsidy_spending(frs_calibrated, calibration_year)
323      calibrate_bus_subsidy_spending(frs_calibrated, calibration_year)
335      calibrate_dataset_fuel_litre_proxies_to_road_fuel(frs_calibrated)
336      calibrate_bus_fare_spending(frs_calibrated, calibration_year)
343      save

Those four calibrations mutate stored columns in place after the down-rating, and rail_usage, rail_subsidy_spending, bus_subsidy_spending and bus_fare_spending are not in storage/uprating_factors.csv, so uprate_dataset leaves them untouched in either direction.

Consequence: uprating the shipped enhanced dataset from its base year back to the calibration year does not recover the dataset the weight calibration was trained on. It yields a third dataset — correct weights and monetary levels, but rail/bus columns carrying post-calibration scalings fitted to calibration-year targets that the weights never saw. The rail and bus calibrations also compute their targets at the calibration year while time_period is temporarily the base year, so their scale factors are year-mixed and not invertible by any uprating factor.

Immaterial for work that does not touch rail, bus or fuel aggregates, but it means "uprate the shipped file to get the calibration-year dataset" is not exact, and it is not documented anywhere.

Suggested fix: run these calibrations before the base-year materialisation, or document that the shipped base-year file is not uprate-invertible to the calibration year.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.