PolicyEngine / PolicyEngine/microcosm
Make target materialization checkpoints reusable across calibration-only runs
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 4
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 94
Description
Problem
Changing calibration settings such as epochs, loss weights, or optimizer options should not rerun expensive PolicyEngine target materialization. Today the fiscal refresh scorer/build path can rematerialize JCT reform vectors such as SALT even when the reform definition itself did not change.
This is especially visible when validating Populace on the full CD target surface: tools/score_us_fiscal_targets.py and tools/build_us_fiscal_refresh_release.py materialize JCT targets again because the cache identity includes broad build state such as build commit and target registry version. That is conservative, but too coarse for launch iteration.
Desired design
Use finer-grained checkpoints:
- Reform-vector cache keyed by the inputs that actually determine per-household reform estimates:
(base_h5_sha256, policyengine_us_version, reform_id, period, congressional_district_crosswalk_sha256 if geography-dependent). Do not include unrelated target registry changes or the whole Populace build commit when the reform code and PE-US version are unchanged. - Target-frame / constraint-matrix cache keyed by
(base_h5_sha256, exact target_surface.sha256, materializer_code_version, policyengine_us_version). - Calibration-only reruns should load the cached target frame/matrix and only solve weights.
Acceptance criteria
- A calibration-only rerun with the same base H5 and target surface does not rerun JCT/SALT reform simulations.
- A target-surface semantic change still invalidates the target frame/matrix safely.
- Two different H5s, such as incumbent vs candidate with different record counts, do not share incompatible per-household vectors.
- Diagnostics/manifest record which checkpoint keys were used.
Context
This came up during the June 2026 full-CD Populace validation. A corrected target-surface score needed fresh materialization, but future changes to only epochs/loss/optimizer should not pay that cost again.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tools/score_us_fiscal_targets.py and tools/build_us_fiscal_refresh_release.py, tracing how target materialization and cache identity are currently handled. Verify the checkpoint keys against the desired inputs, then exercise calibration-only reruns and target-surface or H5 changes to confirm reuse and safe invalidation; diagnostics and manifests should record the keys used.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100