PolicyEngine / PolicyEngine/firm-microsim-paper
Restore VAT-liability-by-sector as a calibration target
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 5h 5m
- Merged PRs (30d)
- 10
Description
Background
vat_liability_sector (HMRC net VAT liability by trade sector) has been removed from the calibration objective — see Config.calibrate_vat_liability_sector = False (gated in firm_microsim/calibration.py). It is still computed and reported as an informational diagnostic in firm_microsim/validate.py, but it no longer feeds the weight optimizer.
Why it was removed
The model fixes firm inputs and sets liability = turnover − input, but does not yet calibrate the input/output tax structure. As a result the sector-level net liability is structurally unhittable — it over/under-shoots for individual sectors (badly for small ones) and competed with the dimensions the paper relies on. Concretely it scored 43.9% (2023-24) and −121.1% (2024-25, sign-broken), dragging the naive 6-dimension mean down even though the five core dimensions calibrate well (~89–90%).
Removing it lets the optimizer focus on the five calibrated dimensions (HMRC turnover bands, ONS population, employment, sector counts, VAT liability by band) and gives a cleaner, defensible headline accuracy.
What restoring it requires
Restoration should come after the input/output calibration work (tracked in #1), specifically:
- Calibrate the input/output tax structure using HMRC
T9 Input and output taxand/or ONS ABS intermediate consumption, so per-sector net liability is reproducible rather than a by-product of unconstrained input draws. - Then re-enable the target by setting
Config.calibrate_vat_liability_sector = Trueand (likely) raisingvat_liability_sector_importance. - Consider flooring the per-sector accuracy metric at 0 (the current sign-aware metric is uncapped and over-penalizes tiny sectors), so the dimension is scored fairly once calibrated.
Acceptance
- With input/output calibration in place,
vat_liability_sectoraccuracy is positive and reasonable (target ≥ ~70%). - Re-enabling it does not materially degrade the five core dimensions.
Related: #1 (broader future calibration extensions).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading issue #1 and the calibration gate in firm_microsim/calibration.py, then inspect the vat_liability_sector diagnostic in firm_microsim/validate.py. Calibrate the input/output tax structure using the cited HMRC or ONS data before re-enabling Config.calibrate_vat_liability_sector. Done means sector accuracy is positive and about 70% or higher without materially reducing the five core dimensions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100