PolicyEngine / PolicyEngine/microcosm

Should *_would_be_qualified columns be derivations rather than QRF targets?

Open
#416 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

The Build M base jetsam (fixed mechanically in #415) exposed a modeling question worth an architecture-week look.

The QBI family (estate_income_would_be_qualified, farm_operations_income_would_be_qualified, farm_rent_income_would_be_qualified, partnership_s_corp_income_would_be_qualified, rental_income_would_be_qualified, self_employment_income_would_be_qualified, sstb_self_employment_income_would_be_qualified, plus business_is_sstb) are 3–6-atom near-categorical columns imputed through the primary PUF QRF chain. A quantile forest is a strange model for a six-level ratio: it can only reproduce the atom set via leaf quantiles, the atoms look like they encode qualification-rule levels applied to the base income column, and the chain pays a full forest per target for what may be a deterministic (or simple conditional-categorical) relationship to columns drawn earlier in the chain.

Questions for the review:

  1. What defined these columns in the retired us-data pipeline — a rule over the base column, or independent survey/PUF signal? (Chesterton fence: find the origin PR before changing anything.)
  2. If rule-derived: they belong in the deterministic-derivations stage (populace#395 canonical ordering), run once on the assembled multispine, not per-source QRF.
  3. If genuinely stochastic: a categorical draw (per-level gate) is cheaper and better-specified than a QRF over atoms.

Not launch-blocking — #415's leaf bound makes the current model tractable and Build M proceeds under it. Filing so the modeling question doesn't vanish behind the memory fix.

🤖 Generated with Claude Code

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the origin PR and the retired us-data pipeline definitions for the listed *_would_be_qualified columns. Then review the deterministic-derivations stage described in populace#395 and the primary PUF QRF chain; done means documenting whether these columns are rule-derived or stochastic and selecting the appropriate modeling path.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, machine-learning
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.