PolicyEngine / PolicyEngine/policyengine.py

PolicyEngineUKDataset.load() raises KeyError: 'person_weight' for directly-supplied UK datasets

Open
#478 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7
Forks
9
Avg merge
15h 51m
Merged PRs (30d)
9

Description

Constructing PolicyEngineUKDataset(name=..., description=..., filepath=..., year=...) against any published UK microdata file raises KeyError: 'person_weight'.

Reproduction (with any policyengine-uk-data Enhanced FRS file):

import policyengine as pe
from policyengine.tax_benefit_models.uk.model import PolicyEngineUKDataset

ds = PolicyEngineUKDataset(name="x", description="x", filepath=PATH, year=2024)
# -> KeyError: 'person_weight'

Cause: load() in src/policyengine/tax_benefit_models/uk/datasets.py assumes person_weight and benunit_weight are already columns. Published UK microdata carries only household_weight at the household level (verified absent in enhanced_frs_2024_25.h5, enhanced_frs_2023_24.h5 and populace_uk_2023.h5); person and benunit weights are derived. create_datasets in the same file already derives them and saves the rewritten file, so the manifest path works while the direct-filepath path does not.

Fix: factor the derivation into a shared helper and call it from load() too, deriving only when the columns are absent.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/policyengine/tax_benefit_models/uk/datasets.py by reading load() and create_datasets(), then reproduce the direct-filepath failure with a published Enhanced FRS file. Trace the existing weight derivation and verify that both direct loading and the manifest path work, including files where the derived columns are already present.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.