PolicyEngine / PolicyEngine/microcosm

Implement in-tree runtime handlers for declared US source-stage operations

Open
#291 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Problem

After the spec-only country packages change (da8e14a), US source stages are declared in source_stages.json but most cannot execute: us_source_operation_handlers() covers only 6 operation kinds (all ACA-related plus support_clip/disaggregate_aggregate_records). Every other stage's core operations have no handler anywhere in the repo:

  • org_wages: fit_labor_market_models — blocks hours-worked inputs (#242, #248)
  • scf_wealth: replace_sentinels, derive, fit_weighted_qrf, head_carry
  • sipp_tips: fit_tip_income_model, zero_when_false
  • meps_esi_premiums: assign_by_plan_type
  • prior_year_income: join, replace_sentinels
  • mortgage_conversion: all three operations
  • acs_rent: aggregate_person_to_household, fit_weighted_imputer, zero_when_false, head_carry
  • vehicle_assets: fit_vehicle_model, fold_into

Consequence: the published base H5s cannot be reproduced from this repo — only recalibrated. Every input-parity issue (#242, #243, #244, #249) implicitly depends on being able to re-run source stages.

Proposal

Implement shared runtime handlers for the declared operation kinds, prioritized by downstream need:

  1. fit_labor_market_models (org_wages) — hours inputs are the SNAP work-requirements blocker
  2. fit_weighted_qrf + head_carry + replace_sentinels/derive (scf_wealth, acs_rent share these)
  3. The remaining stage-specific fits

The immigration/SSN source stage (#266) established the pattern: manifest entry + shared runtime handler + tests. Handlers should be generic interpreters of manifest parameters (like the existing ACA handlers), not country-specific Python.

Context

Found while auditing SNAP work-requirements readiness: the manifest declares weekly_hours_worked_before_lsr as an org_wages output with full provenance, but nothing in-tree can produce it.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with source_stages.json and us_source_operation_handlers() to map the declared operation kinds and existing ACA coverage. Read the immigration/SSN source-stage implementation and its tests for the established manifest-and-handler pattern. Done means the prioritized declared operations have shared runtime coverage and tests sufficient to reproduce the affected source stages.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.