PolicyEngine / PolicyEngine/policybench

Child slot labels should be youngest-first, not oldest-first

Open
#50 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2
Forks
3
Avg merge
12h 7m
Merged PRs (30d)
13

Description

Problem

scenarios.py:1160-1163 (US) and the analogous UK code at scenarios.py:1337-1357 sort household members by descending age before assigning child1, child2, ... labels:

household.sort_values(
    by=["is_tax_unit_head", "is_tax_unit_spouse", "age", "person_id"],
    ascending=[False, False, False, True],  # age=False → DESCENDING
)

So child1 is the oldest child in the household, child2 the next oldest, etc. This inverts the natural reading for benefit programs that target young children — WIC (age < 5), Head Start (3–4), Early Head Start (under 3), Medicaid/CHIP eligibility (varies by age/income), school-meal eligibility.

Concrete evidence

Across the 10 US multi-child benchmark scenarios:

Scenario child1 age child2 age child3 age ...
008 4 0
030 16 14 11
053 7 5 0
055 17 14 12 10, 5, 1
061 7 3
089 7 3

WIC eligibility (age < 5) by child slot in the sample:

  • child1_wic_eligible: 1 case (only scenario 008)
  • child2_wic_eligible: 3 cases (008, 061, 089)
  • child3_wic_eligible: 1 case (053)
  • child6_wic_eligible: 1 case (055)

The household-impact weight share follows the same shape — child2_wic_eligible weight (0.0017) is ~2× child1_wic_eligible (0.0009). That's the inverse of what users will expect when reading "Child 1 WIC eligibility" in the prompt or the program breakdown table.

Why this matters

  • Prompt readability: a reader expects "Child 1" to be the household's most prominent young-child case for these programs, not the oldest sibling who's usually outside the eligibility age band.
  • Person-level slot weights: programs that key on young children (WIC, EHS) get most of their weight on child2/child3 rather than child1, distorting the program breakdown table.
  • Failure-mode attribution: when a model misses WIC eligibility on the youngest child, the failure currently surfaces under child2_wic_eligible etc., not under the "primary" label.

Proposed fix

Switch the age sort to ascending — youngest child first. After the head/spouse rows, append children oldest-last so child1 is the youngest. Same in the UK path.

- ascending=[False, False, False, True],
+ ascending=[False, False, True, True],

This is a content change that renames many prompt-visible labels (child1_* → was-childN_*), so it will:

  • Invalidate the frozen paper snapshot's reproducibility against re-generated scenarios
  • Shift household-impact weights between childN slots
  • Require updating model outputs that referenced specific child slots by index (audit notes, case annotations)

If the frozen paper snapshot needs to stay byte-identical, this should land alongside a new snapshot date and a paper note that the convention changed.

Out of scope

  • Headline metric / weighting from the population — separate issue (#49).
  • Adult slot ordering (adult1, adult2) — currently spouse comes second by is_tax_unit_spouse flag; that's a different sort and isn't affected by this change.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in scenarios.py:1160-1163 for the US path and scenarios.py:1337-1357 for the UK path, then inspect the generated scenarios and frozen paper snapshot for child-slot references. Change the ordering so child1 is youngest, regenerate affected artifacts, and update model outputs, audit notes, and case annotations; done means both paths and downstream references consistently use the new convention.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.