PolicyEngine / PolicyEngine/microcosm

ORG full-year equivalence is a proxy, not a concept identity: claims + validation need alignment (#539 follow-up)

Open
#544 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Post-merge audit finding on #539 (6e462ca), cross-family (sol adversarial pass with survey-definition citations). Severity: HIGH, concept integrity (no-fabricated-mechanisms class).

Defect

The transform computes annual wage income × 52 / weeks_worked and describes the result as matching an "annualized reference week" donor concept (org_wages.py ~:591/:603; source_stages.json ~:1517). That equivalence does not follow from the survey definitions:

  • ORG PTERNWA is usual weekly earnings at the main job (multiple-job holders: main job only — BLS CPS definitions).
  • ASEC WSAL_VAL aggregates wage/salary earnings across employers; WKSWORK counts weeks with any work, including paid leave (ASEC 2024 data dictionary).
  • So the claimed equivalence holds only for stable, single-wage-job workers; it is an approximation for multiple-job, mixed wage/self-employment, and changing-earnings cases.
  • The PUF channel is weaker still: PUF tax-unit income totals are allocated over cloned ASEC people while ASEC weeks are retained (puf_support.py ~:654/:1066/:1678); dividing allocated PUF income by cloned ASEC weeks has no demonstrated work-duration interpretation.

What held up under audit (keep): masked division handles zero/missing weeks; donor zero-hours filtered; no calibration-target coupling (pre-calibration feature path only); QRF seeding explicit; suite green on head.

Required

  1. Rewrite the exact/concept-clean claims as what the transform is: a proxy with a stated validity domain (docstrings, stage description, PR-body-derived docs).
  2. Add validation/tests for the semantic cases the claim glosses: multiple-job holders, mixed wage/SE income, PUF-supported records; plus the untested clipping edges (negative, >52 weeks).
  3. Or: source a genuinely aligned recipient-side measure and make the equivalence real.

Owner: the org-wage-weeks lane (#529 / #539 authors). Related operational finding filed separately (materializer version bump).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read the transform and claims in org_wages.py around lines 591 and 603, then compare its stage description in source_stages.json around line 1517. Inspect the PUF allocation paths in puf_support.py around lines 654, 1066, and 1678. Done means the documentation calls the result a proxy with a validity domain and validation covers multiple-job, mixed wage/self-employment, PUF-supported, and clipping cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.