PolicyEngine / PolicyEngine/policyengine.py
PolicyEngineUKDataset.load() raises KeyError: 'person_weight' for directly-supplied UK datasets
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7
- Forks
- 9
- Avg merge
- 15h 51m
- Merged PRs (30d)
- 9
Description
Constructing PolicyEngineUKDataset(name=..., description=..., filepath=..., year=...) against any published UK microdata file raises KeyError: 'person_weight'.
Reproduction (with any policyengine-uk-data Enhanced FRS file):
import policyengine as pe
from policyengine.tax_benefit_models.uk.model import PolicyEngineUKDataset
ds = PolicyEngineUKDataset(name="x", description="x", filepath=PATH, year=2024)
# -> KeyError: 'person_weight'
Cause: load() in src/policyengine/tax_benefit_models/uk/datasets.py assumes person_weight and benunit_weight are already columns. Published UK microdata carries only household_weight at the household level (verified absent in enhanced_frs_2024_25.h5, enhanced_frs_2023_24.h5 and populace_uk_2023.h5); person and benunit weights are derived. create_datasets in the same file already derives them and saves the rewritten file, so the manifest path works while the direct-filepath path does not.
Fix: factor the derivation into a shared helper and call it from load() too, deriving only when the columns are absent.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/policyengine/tax_benefit_models/uk/datasets.py by reading load() and create_datasets(), then reproduce the direct-filepath failure with a published Enhanced FRS file. Trace the existing weight derivation and verify that both direct loading and the manifest path work, including files where the derived columns are already present.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100