PolicyEngine / PolicyEngine/microcosm

UK: supply the engine's local_authority input from the ladder's local_authority_code, plus council tax follow-ups from policyengine-uk#1855/#1864 and uk-data#482/#484

Open
#953 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Summary

Every household row in the UK build carries an ONS local authority code (local_authority_code, written by the geography ladder in uk_runtime/geography_ladder.py, April 2023 LAD vintage, 361 areas), but nothing maps it into the engine's local_authority enum, and the policyengine-uk adapter only supplies engine inputs that exist as frame columns. So the engine sees the enum default (MAIDSTONE) for every household in every Microcosm build, exactly as it does on the uk-data datasets.

This is the one council-tax gap Microcosm can close and uk-data structurally cannot: uk-data reweights one national sample per authority, so a household-level enum can never vary there; Microcosm assigns an authority per row. It matters now because:

  • PolicyEngine/policyengine-uk#1855 adds council_tax_band_d_amount (an LA-keyed parameter read through household("local_authority")) and simulated_council_tax; on any dataset without the input they collapse to Maidstone's Band D times the national band mix (the £61.21bn England artefact on that PR).
  • The bespoke Council Tax Reduction schemes in variables/gov/local_authorities/council_tax_reduction/config.py test local_authority == LocalAuthority.MERTON (and Kingston upon Thames, Newham, Oxford, Westminster). They never fire on any dataset today.
  • The CT-C reconciliation on #736 (rules-based liability from band, the authority's Band D level and the statutory ratios, with CTR engine-side) is unmeasurable until the engine knows each row's authority.

Review context: PolicyEngine/policyengine-uk#1855 (changes requested 2026-09-18), PolicyEngine/policyengine-uk#1864, PolicyEngine/policyengine-uk-data#482, PolicyEngine/policyengine-uk-data#484, TheAxiomFoundation/rulespec-uk#346, tracker PolicyEngine/policyengine-uk#1854.

Deliverable A: supply local_authority as an engine input

Design, following the region precedent (FRS gvtregno mapped to enum key strings such as SOUTH_EAST on the household table; council_tax_band is supplied the same way):

  1. Names source. The ladder sources carry codes only (no LAD23NM-style column). Vendor a hash-pinned ONS names-and-codes lookup for the three roster vintages the crosswalk declares (ew:2023_april_lad, scotland:2019_council_area, ni:2014_lgd), the way geography_sources.py pins its other lookups.
  2. Key derivation. The engine enum key is a mechanical function of the ONS display name for 378 of its 379 members (uppercase, non-alphanumerics to single underscores, e.g. Bristol, City of to BRISTOL_CITY_OF, Na h-Eileanan Siar to NA_H_EILEANAN_SIAR). The single exception is King's Lynn and West Norfolk (KINGS_LYNN_AND_WEST_NORFOLK, apostrophe dropped). So: mechanical rule plus a one-entry alias table, not a hand-written 361-row map.
  3. Fail closed. A build-time test resolves every local_area_crosswalk.json area id to a LocalAuthority member at the pinned engine version and refuses on any miss. No silent fall-through to the enum default.
  4. Write point. Immediately after the ladder assignment (the column is per row, so clones already differ), consistent with region_code by construction since authorities nest in regions. Add local_authority to uk/release_input_coverage_manifest.json as required.

Dependency. The engine enum is the 2021 LAD list and lacks the six April 2023 unitaries (North Northamptonshire, West Northamptonshire, Cumberland, Westmorland and Furness, North Yorkshire, Somerset), which are exactly the codes on our roster with no member. Adding them is requested on PolicyEngine/policyengine-uk#1855; if it lands elsewhere, link it here. Until the pinned engine carries them the fail-closed test blocks the stage, which is the intended behaviour (a partial map would misprice about 1.4 million people silently). Northern Ireland's 11 districts resolve to the enum's members and correctly get zero council tax.

Acceptance.

  • Every household row resolves; the test above is green at the pinned engine.
  • On a build with the engine pin at or past PolicyEngine/policyengine-uk#1855: distinct council_tax_band_d_amount values per nation equal the authority count (296, 22, 32); the five bespoke CTR schemes show non-zero caseloads; the England simulated_council_tax aggregate is reported next to the PR's single-authority figure as a receipt, not bound.

Other follow-ups these PRs expose

  • Parity register housekeeping after PolicyEngine/policyengine-uk-data#482 and #484 merge. Both change targets/sources/la_council_tax.py and tests/test_la_council_tax_targets.py; uk/uk_data_target_inventory.json pins sha256s for both, so verify_uk_data_target_inventory_against_tree refuses until re-pinned. Rewrite local_council_tax_band_d_rate.fence.origin (it says uk-data "did not wire them into the additive LA loss matrix"; the emission existed and was silently dropped until #484) and cite PolicyEngine/policyengine-uk-data#483 and #484 in its evidence. credibility_local_council_tax_outliers covers the same test file and inherits the drift.
  • Per-band amounts (#482): nothing to port, say so. All 296 English authorities' bands A to H are exact ninths of Band D to within half a penny, so the eight new columns carry nothing beyond the Band D fact already pinned in Chronicle (mhclg/council_tax_levels_england_2026_27, fiscal 2025 and 2026) and the ratio parameter. Extend the local_council_tax_band_d_rate reason to name the eight columns so nobody ports them as targets.
  • Single-adult discount family candidate. mhclg.council_taxbase.single_adult_discount_dwellings (CTB line 8, per authority, already in the pinned feed) is a count, so it is additive and a legitimate local target once PolicyEngine/policyengine-uk#1864 lands and Deliverable A is in. Measurement filter: the spine's council_tax_single_adult_raw == 1 (FRS adulth, currently written and never consumed) or the engine's council_tax_discount_rate. England first; check the Welsh CT1 and Scottish equivalents before declaring a family.
  • CT-C measurement conditions (tracked on #736, recorded here for the dependency). Needs an engine pin with fiscal 2025 Band D values (requested on PolicyEngine/policyengine-uk#1855; today it carries 2026-27 only for England and Wales). Compare simulated_council_tax_less_discounts against the FRS council_tax input per authority, because FRS reports the billed amount (net of discounts, gross of CTR). Do not replace the council_tax input with the computed value; it feeds council_tax_less_benefit and the CTR stack. Note the cross-nation base mix on #1864 (discount zero outside England) until its scope is settled.
  • Axiom adapter naming. TheAxiomFoundation/rulespec-uk#346 queues the LGFA 1992 charging sections; when modules are generated, adapters/axiom.py is where they meet the policyengine encoding. Names and entity level should track council_tax_band, local_authority, simulated_council_tax, council_tax_discount_rate (household). Nothing to do until then.

Already tracked elsewhere

  • CT-B per-clone band redraw and the band-H spine support deferral: #796.
  • CT-C rules-based liability, OBR nation rows report-only, council_tax_net blocked on its source: #736.
  • Taxbase-basis band-count family: #934 (merged).

🤖 Generated with Claude Code

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with uk_runtime/geography_ladder.py, geography_sources.py, local_area_crosswalk.json, and uk/release_input_coverage_manifest.json; trace the existing region and council_tax_band input assignments. Add the pinned authority-name lookup, key derivation and alias handling, then make the build-time crosswalk resolution fail closed and verify every household row resolves at the required engine version.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, data-engineering, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.