PolicyEngine / PolicyEngine/calibration-diagnostics

Evaluation harness: support aged targets (declared fact-ratio levels) in precomputed checkpoints

Open
#171 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
0
Forks
4
Avg merge
1d 9h
Merged PRs (30d)
13

Description

Found publishing Microcosm-BE v0.5 (2025 simulation) to the cross-dataset page. v0.5's release surface contains 233 aged targets — 2025 levels derived as published fact × declared FPB fact-ratio with both record ids and the factor in the manifest aging block. The precomputed-checkpoint adapter (#164) enforces two constraints that are jointly unsatisfiable for such releases:

  1. every checkpoint row's period must equal its Chronicle facts' period (precomputed checkpoint row 'statbel_b1_decile_01_total_tax' period does not match its Chronicle facts — the row is a 2025 level over …income_year2023… facts);
  2. the checkpoint row set must cover the authoritative Microcosm release surface (filtering the aged rows out fails with differs from the authoritative Microcosm release surface; missing=[233 rows]).

The US path solves this with first-class aging transforms (microcosm_aging, period treatment aligned_fact). The BE precomputed path needs the equivalent: accept a per-row alignment declaration (numerator/denominator fact ids + factor, mirroring the release manifest's aging block), verify the arithmetic against the named facts, and surface it as the existing aligned_fact period treatment with calibration_exposure intact — never silently accepting a mismatched period.

Until then the datasets page stays on the v0.4 run (evaluation-f28ca06a…, unaged surface) while the calibration-fit page serves v0.5; the v0.5 checkpoints are in population-rerun/out/v05b/evaluation/ ready to re-run once supported.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the precomputed-checkpoint adapter referenced in #164 and compare it with the US microcosm_aging path. Inspect the release manifest’s aging block and the checkpoints in population-rerun/out/v05b/evaluation/. Done when the v0.5 checkpoints can be rerun with verified aligned facts, preserved calibration_exposure, and no silent period mismatch.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
data, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.