PolicyEngine / PolicyEngine/policyengine.py
ensure_datasets: consume Microcosm-published UK releases (manifest-verified), so analyses can swap bases by URI
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7
- Forks
- 9
- Avg merge
- 15h 51m
- Merged PRs (30d)
- 9
Description
Doctrine says policyengine.py is the permanent interface and Microcosm is where UK data construction is migrating (PolicyEngine/microcosm#665). The consumption path is currently missing: pe.uk.ensure_datasets materialises per-year files from hf://policyengine/policyengine-uk-data/..., and there is no way to point it at a Microcosm-published UK release (certified single-year artifact + release manifest with pinned SHA-256s).
Concrete need, with a live consumer waiting: PolicyEngine/uk-cgt-paper#6 and PolicyEngine/uk-equalising-cgt#1 — both reform analyses should rerun on the Microcosm UK build once it certifies (currently gated on microcosm#630), ideally changing nothing but a dataset URI.
Scope sketch:
- accept a Microcosm release identifier (HF repo + release id) in
ensure_datasets; - verify artifact SHA-256 against the release manifest before use (Microcosm releases pin these; the interface should check, matching Microcosm's own runtime verification discipline);
- resolve the release's
compatible_model_packages/compatible_core_packagesconstraints and fail loudly on mismatch with the installed bundle; - forward-year materialisation (uprating 2023 → simulation years) with the same semantics the uk-data path applies today, so results are comparable across the swap.
Related: #491 (publisher claim). Happy to pair on the design — the UK CGT arc (microcosm#552) is the first artifact that wants this path.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the pe.uk.ensure_datasets entry point and trace the existing hf://policyengine/policyengine-uk-data materialisation and forward-year uprating path. Compare it with the Microcosm release manifest and its compatible_model_packages and compatible_core_packages constraints; done means a release URI works with SHA-256 and bundle checks while preserving current semantics for the UK CGT consumer analyses.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100