PolicyEngine / PolicyEngine/calibration-diagnostics
Evaluation harness: support aged targets (declared fact-ratio levels) in precomputed checkpoints
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 0
- Forks
- 4
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 13
Description
Found publishing Microcosm-BE v0.5 (2025 simulation) to the cross-dataset page. v0.5's release surface contains 233 aged targets — 2025 levels derived as published fact × declared FPB fact-ratio with both record ids and the factor in the manifest aging block. The precomputed-checkpoint adapter (#164) enforces two constraints that are jointly unsatisfiable for such releases:
- every checkpoint row's
periodmust equal its Chronicle facts' period (precomputed checkpoint row 'statbel_b1_decile_01_total_tax' period does not match its Chronicle facts— the row is a 2025 level over…income_year2023…facts); - the checkpoint row set must cover the authoritative Microcosm release surface (filtering the aged rows out fails with
differs from the authoritative Microcosm release surface; missing=[233 rows]).
The US path solves this with first-class aging transforms (microcosm_aging, period treatment aligned_fact). The BE precomputed path needs the equivalent: accept a per-row alignment declaration (numerator/denominator fact ids + factor, mirroring the release manifest's aging block), verify the arithmetic against the named facts, and surface it as the existing aligned_fact period treatment with calibration_exposure intact — never silently accepting a mismatched period.
Until then the datasets page stays on the v0.4 run (evaluation-f28ca06a…, unaged surface) while the calibration-fit page serves v0.5; the v0.5 checkpoints are in population-rerun/out/v05b/evaluation/ ready to re-run once supported.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the precomputed-checkpoint adapter referenced in #164 and compare it with the US microcosm_aging path. Inspect the release manifest’s aging block and the checkpoints in population-rerun/out/v05b/evaluation/. Done when the v0.5 checkpoints can be rerun with verified aligned facts, preserved calibration_exposure, and no silent period mismatch.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- data, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100