PolicyEngine / PolicyEngine/microcosm

Validate populace-be against EUROMOD-BE and Federal Planning Bureau scores

Open
#264 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Part of #259 (populace-be epic).

What

Belgium has no incumbent dataset inside our stack, so the US/UK self-parity gate is replaced by external oracles. Two tracks:

  1. Baseline comparison vs EUROMOD-BE. EUROMOD is open-access; where the SILC licence permits running it on the same input vintage, compare baseline outputs: poverty rates (AROP), income decile means, PIT revenue, social-contribution totals, main benefit aggregates. Where we cannot run it, compare against published EUROMOD-BE country-report statistics. Document every comparison's vintage and concept bridge.
  2. Reform validation vs Federal Planning Bureau costings. The JCT/OBBBA analog: a curated list of Belgian reforms with official published scores (BFP analyses; SPF Finances estimates where available), split in-sample (targets the calibration saw) vs out-of-sample (never seen). Wire into the reform_validation.json release contract with the existing out_of_sample_simulated requirement.

Dependencies

  • Reform runs need counterfactual materialization through the Axiom adapter — sequenced behind the adapter issue's reform-extension decision.
  • Rules-level (single-household) cross-engine validation against EUROMOD/other calculators belongs to the Axiom oracles work (TheAxiomFoundation/axiom-oracles); this issue is the dataset-level analog — population aggregates and distributions, not per-case rule correctness.

Acceptance

  • A release ships reform_validation.json against named BFP/EUROMOD-BE comparators for the encoded slice, with in/out-of-sample labeling and source URLs per row.
  • Baseline comparison table (populace-be vs EUROMOD-BE vs admin) published in the release diagnostics.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the existing release contract and the reform_validation.json output, then review the Axiom adapter dependency before planning the reform runs. Define the named EUROMOD-BE, BFP, and administrative comparators, their vintage and concept bridges, and the in- and out-of-sample labels; done means the release diagnostics contain the baseline table and a sourced reform_validation.json for the encoded slice.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, release, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.