PolicyEngine / PolicyEngine/microcosm

By-origin battery design adjudication

Open
#783 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Adjudication needed

The 25% candidate run and an adversarial feasibility audit (2026-08-26) surfaced design questions in the by-origin battery that need owner rulings — distinct from genuine model drift, which is being root-caused separately:

  1. Max-of-five quantile envelope false-block rate. Simulating the envelope test against the fixed ASEC empirical distribution at each leg's realized effective support, six person legs fail more often than they pass even under a faithful null: qualified BDC income (24% pass), farm income (30%), W-2 wages from qualified business (36%), Form 4952 (39%), casualty loss (44%), negative estate income (48%). This is not purely a low-count problem — farm income has amount-leg ESS ≈ 1,480 and still 30%, because discrete mass near a tested quantile destabilizes a max-of-five rule.
  2. Authority per leg. Some references are themselves imputed (e.g. alimony_expense: ASEC does not measure alimony paid; the ASEC-side reference is PUF-imputed — the comparison grades imputation against imputation). Others are cross-source clashes (alimony_income: survey-measured ASEC 0.056% vs PUF-fitted draws — the live incumbent's own asec/puf channels disagree 2.6× on the same leg). The battery currently treats ASEC-origin as truth for all legs.
  3. Family-level error control. 135 correlated comparisons with per-comparison fatal thresholds; no multiplicity treatment.
  4. What the incidence data does NOT support: a generic rare-leg support floor. At 25% support, no failing incidence leg is a luck problem (lowest ideal-null pass probability ≈ 86%); 19 of 26 failing incidence intervals sit wholly outside the band. Incidence failures are systematic and should be root-caused, not floored.

Companion issues cover the two deterministic clone-0 QBI comparisons and the weeks-unemployed infeasible constraint. Ruling should fix the test's design integrity without tuning anything toward passing a specific artifact.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the by-origin battery and its max-of-five envelope, authority-per-leg, and family-level error-control logic, then compare the incidence findings with the companion QBI and weeks-unemployed issues. Done means an owner ruling resolves the four listed design questions and preserves design integrity without tuning toward a specific artifact.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, testing-qa
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.