PolicyEngine / PolicyEngine/policyengine-uk
Imputed potential earnings are too coarse to exercise the OBR participation elasticity gradient
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 50
- Forks
- 33
- Avg merge
- 20h 58m
- Merged PRs (30d)
- 14
Description
Summary
impute_wages_for_nonworkers assigns every non-worker one of about a dozen sex-and-age-band medians. After #1829 places them against the observed worker earnings distribution on a full-time-equivalent basis, they occupy only quintiles 3 and 4 on enhanced_frs_2024_25 — none reach 1, 2 or 5.
That matters because the quintile exists solely to index the OBR Table A1 participation elasticities, whose values vary up to tenfold across quintiles. For the population whose entry the module models, that gradient is barely exercised.
Measured
Weighted quintile shares of non-working adults, on the merged state of #1829:
| quintile | share of non-working adults |
|---|---|
| 1 | 0.000 |
| 2 | 0.015 |
| 3 | 0.622 |
| 4 | 0.364 |
| 5 | 0.000 |
Working adults, by contrast, split 0.198 / 0.201 / 0.200 / 0.201 / 0.200 — the thresholds themselves are fine.
History, so the cause is not misattributed
Before #1829 the problem was worse and different: quintiles were taken over the whole population including children on actual earnings, so the bottom two quintiles contained no earners at all and 26,711 non-working adults were imputed a wage of exactly zero — which silently bars entry, since apply_participation_responses gates on imputed_wages[i] > 0.
#1829 fixed that. It then surfaced a second issue: imputed earnings were computed at 18.8 hours a week while thresholds came from mostly full-time observed earnings, so every imputed value fell inside a single threshold interval. Scaling to full-time equivalent moved them off that interval — to two intervals.
So this is not a placement problem. It is that the imputation has only about a dozen distinct values.
What would fix it
A finer imputation, not a different placement rule. Options, roughly in order of effort:
- Narrower age bands. The current
np.clip(age // 10 * 10, 10, 60)pools ages 0-19 into one band and 60-80 into another — the two doing the most extrapolation are the least homogeneous. - Condition on more than sex and age: region, education, previous earnings where observed, presence and age of children.
- Impute a distribution rather than a group median, so within-group variation survives.
Not urgent
apply_participation_responses is commented out of its coordinator (dynamics/labour_supply.py:157), so nothing in production reads this today. It should be resolved before the participation model is enabled, because it systematically moves non-workers from the high-elasticity end of Table A1 to the middle, and the entrant count is sensitive to it.
Found in adversarial review of #1829.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading impute_wages_for_nonworkers and the participation path in dynamics/labour_supply.py, including the coordinator around line 157. Reproduce the weighted quintile shares on enhanced_frs_2024_25 and compare the available imputation options. Done means selecting and validating a finer imputation that gives non-workers meaningful coverage across the OBR participation-elasticity quintiles without restoring zero-wage exclusions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100