PolicyEngine / PolicyEngine/policyengine-api

Migrate policyengine dep from 0.x to v4 (prereq for TRACE TRO emission)

Open
#3,486 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
18
Forks
33
Avg merge
23h 40m
Merged PRs (30d)
22

Description

Context

While scoping PolicyEngine/policyengine-api#3485 (TRACE TRO emission for webapp simulation runs) I found that policyengine-api is pinned to policyengine>0.12.0,<1 and imports modules from the pre-v4 orchestrator that do not exist in current pe.py (4.3.1+):

  • policyengine.simulation.SimulationOptions (used in policyengine_api/services/economy_service.py:23)
  • policyengine.utils.data.datasets.get_default_dataset (used at policyengine_api/services/economy_service.py:24)

Both were removed during the 0.x → 4.x rearchitecture. The new v4 provenance primitives we need for TRACE TRO emission (policyengine.provenance.trace.build_trace_tro_from_release_bundle, build_simulation_trace_tro) only exist on the v4 line.

What needs to happen

Migrate the api from policyengine 0.x → 4.3.1+. At a minimum:

  • Replace SimulationOptions usages with the v4 equivalent (whatever the v4 unified orchestrator exposes for simulation configuration). See src/policyengine/simulation.py in the policyengine.py repo for the current shape.
  • Replace get_default_dataset usages with the v4 dataset resolution path. See src/policyengine/tax_benefit_models/common/model_version.py and src/policyengine/provenance/manifest.py for how v4 resolves certified data.
  • Bump policyengine-us → 1.653.3+ and policyengine-uk → 2.88.0+ (the versions v4 is certified against).
  • Update any other internal APIs that changed between v0.x and v4.
  • Re-run the full integration test suite; simulation output numbers should not change if the migration is done correctly (v4 is a thin orchestrator over the same country packages; model math is unchanged).

Why this is a prerequisite

TRACE TRO emission requires calling pe.py v4's provenance helpers, and vendoring those into the api would fork the canonical JSON bytes (hashes would not match). The migration therefore needs to land before #3485 is implementable.

Scope

This issue is scoped to just the version bump and the pre-v4 → v4 API mapping. Not scoped: any redesign of how the api sets up simulations, any new features, any refactor beyond what's needed to get the test suite green on v4.

Related

  • PolicyEngine/policyengine-api#3485 (TRO emission — blocked on this)
  • policyengine.py release notes for 4.0–4.3.1 (breaking changes)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in policyengine_api/services/economy_service.py, then compare its SimulationOptions and get_default_dataset usage with src/policyengine/simulation.py, src/policyengine/tax_benefit_models/common/model_version.py, and src/policyengine/provenance/manifest.py in policyengine.py. Review the 4.0–4.3.1 release notes and run the full integration test suite. Done means the v4 migration and required country-package bumps are complete, the suite is green, and simulation output numbers are unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.