PolicyEngine / PolicyEngine/policyengine-uk-data
Rail, bus and fuel calibrations run after base-year materialisation, so the shipped file is not uprate-invertible
@juaristi22 is already working on this.
Since Sep 3, 2026.
- Dominant language
- Python
- Stars
- 1
- Forks
- 2
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 4
Description
_materialize_base_year_dataset is not the final transformation before the dataset is saved. In create_datasets.py:
304-311 _materialize_base_year_dataset -> uprate_dataset(frs_calibrated, base_year)
319 calibrate_rail_subsidy_spending(frs_calibrated, calibration_year)
323 calibrate_bus_subsidy_spending(frs_calibrated, calibration_year)
335 calibrate_dataset_fuel_litre_proxies_to_road_fuel(frs_calibrated)
336 calibrate_bus_fare_spending(frs_calibrated, calibration_year)
343 save
Those four calibrations mutate stored columns in place after the down-rating, and rail_usage, rail_subsidy_spending, bus_subsidy_spending and bus_fare_spending are not in storage/uprating_factors.csv, so uprate_dataset leaves them untouched in either direction.
Consequence: uprating the shipped enhanced dataset from its base year back to the calibration year does not recover the dataset the weight calibration was trained on. It yields a third dataset — correct weights and monetary levels, but rail/bus columns carrying post-calibration scalings fitted to calibration-year targets that the weights never saw. The rail and bus calibrations also compute their targets at the calibration year while time_period is temporarily the base year, so their scale factors are year-mixed and not invertible by any uprating factor.
Immaterial for work that does not touch rail, bus or fuel aggregates, but it means "uprate the shipped file to get the calibration-year dataset" is not exact, and it is not documented anywhere.
Suggested fix: run these calibrations before the base-year materialisation, or document that the shipped base-year file is not uprate-invertible to the calibration year.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.