PolicyEngine / PolicyEngine/policyengine-scorecard

Assumptions registry: catalog every source model's methodology for apples-to-apples comparison

Open
#10 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
0
Avg merge
6d 12h
Merged PRs (30d)
32

Description

Max (8/2): the goal is to present comparisons, understand model differences, and find opportunities to improve PE by understanding them — not blindly adopting others' practices, but potentially adding capabilities. "We should also catalog the detailed assumptions like any elasticities they might be applying, how they bin for deciles etc, so we can make the most apples to apples comparison."

The artifact

A structured model_assumptions registry (DB table + per-source YAML in sources/<source>/assumptions.yaml), one record per (source_model, dimension), with verbatim provenance to the shop's own methodology publication. Dimension vocabulary (closed, fail-loud, extend deliberately):

  • income_concept (TPC expanded cash income / PWBM after-tax-and-transfer / TF market income / CBO household income…)
  • distribution_unit (tax units / households / persons) + equivalence_adjustment + negative_income_handling + percentile_thresholds
  • elasticity_labor_supply, elasticity_cg_realization, other behavioral parameters (values + vintage)
  • behavioral_scope (static / conventional-with-microdynamic / dynamic-macro), scoring_convention
  • corporate_incidence_split, baseline_convention, data_base (PUF year, CPS match), aging_method, takeup_assumptions
    Second harvest pass: methodology briefs per shop (TPC model description, PWBM methodology pages, TF Taxes-and-Growth docs, JCT macro-analysis JCX docs, CBO methods write-ups) — small artifact count, high value.

What it powers

  1. Apples-to-apples runs: per comparison row, machine-readable assumptions_matched vs assumptions_native. Where PE machinery allows, add assumption-matched PE runs (re-bin by their income concept — tabulation-layer; set LSR elasticities to their values — existing parameters; corporate incidence — cannot, row says so). Feeds the same_assumptions benchmark class (#9).
  2. Structured decomposition: "explained by assumption delta" links two registry entries instead of prose.
  3. Capability queue with the non-adoption filter: divergence traced to an assumption PE cannot vary → a capability issue in pe-us/populace (support it as an option), explicitly NOT a default to adopt it. Each such issue cites the registry entries + the divergences it would explain.

PE's own record goes in the registry too (engine version-pinned) — we're a row in our own catalog, same standard as everyone else.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the proposed model_assumptions database table, the closed dimension vocabulary, and the per-source sources/<source>/assumptions.yaml artifacts. Catalog each source model and the engine-versioned PE record with verbatim methodology provenance, then document methodology briefs and expose matched versus native assumptions for comparisons. The issue does not name a test or runnable entry point.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.