PolicyEngine / PolicyEngine/policyengine-scorecard
Take-up assumptions: compare PolicyEngine and UKMOD where they are comparable
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 6d 12h
- Merged PRs (30d)
- 32
Description
Found while researching UKMOD (CeMPA/Essex) for scorecard comparison. Verified by reading the take-up parameters out of a real policyengine-uk 2.89.2 — the certified engine — not from documentation.
The finding
PolicyEngine-UK's take-up parameters, read from the engine:
| parameter | value | dated from |
|---|---|---|
gov/dwp/housing_benefit/takeup |
1.0 | 2019-01-01 |
gov/dwp/income_support/takeup |
1.0 | 2019-01-01 |
gov/dwp/tax_credits/child_tax_credit/takeup |
1.0 | 2019-01-01 |
gov/dwp/tax_credits/working_tax_credit/takeup |
1.0 | 2019-01-01 |
gov/dwp/universal_credit/takeup_rate |
0.55 | 2015-01-01 |
gov/dwp/pension_credit/takeup |
0.70 | 2015-01-01 |
gov/dwp/JSA/income/takeup |
0.67 | 2012-04-06 |
UKMOD, for the same benefits, applies non-take-up probabilities sourced from DWP (2020) "Income-related benefits: estimates of take-up" and HMRC (2019) tax credits take-up, varying by claimant circumstance (UKMOD Country Report 2023–2030, CeMPA WP 8/26, Table 3.4):
| benefit | UKMOD assumption |
|---|---|
| Housing Benefit | 0.85 pension-age · 0.57 working-age in work · 0.96 working-age not in work |
| Income Support | 0.89 without children · 0.92 with children |
| CTC + WTC | 0.95 lone parents (non-London) · 0.73 couples with children · 0.32 WTC no children |
| Pension Credit | 0.69 guarantee · 0.37 savings-only |
So on four legacy benefits PolicyEngine assumes everyone entitled claims, where the published DWP/HMRC evidence says between 32% and 96% do, depending on who they are.
Why it matters for the scorecard
Two distinct consequences, and I want to be careful not to overstate either:
-
A caseload or expenditure comparison for HB/IS/CTC/WTC is not comparing the same quantity. PolicyEngine is producing an entitlement count; DWP's administrative figure is a receipt count. Any lane scoring those against each other is measuring non-take-up and attributing it to the engine. This is exactly the failure
GAP_REASONSand the divergence registry (#59) exist to name — but nothing currently names it. -
Calibration may be silently absorbing it. PolicyEngine reweights to DWP/OBR caseload and expenditure targets. If entitlement is simulated at 100% take-up and then weighted to match a receipt-based total, the weights are compensating for a modelling assumption. The scorecard already distinguishes this — OBR rows carry
calibration_relationship = consumed_as_target— but the take-up axis isn't represented at all.
Proposed lane
A parameter-level comparison rather than an output one, which the scorecard has none of today: record PolicyEngine's take-up parameters read live from the certified engine, alongside UKMOD's published assumptions, with the originating DWP/HMRC source cited for each.
It stages no value claims (the #86/#91 rule — the rates originate with DWP and HMRC, and UKMOD's table is a transcription of them plus mid-point choices of its own). What it records is: what each model assumes, where the assumption came from, and how stale it is.
Caveats worth stating up front
- UKMOD applies take-up at household level per benefit with a randomisation rule; PolicyEngine's are national scalars. These are not the same mechanism, so the numbers are not strictly like-for-like — the comparable object is the assumed aggregate rate, not the algorithm.
- PolicyEngine does have reported-recipient anchoring in
policyengine-uk-data, so real behaviour may be closer than the bare parameter suggests. That should be checked before any conclusion is drawn about output error. - UKMOD flags its own rates as stale: DWP/HMRC have not updated working-age take-up statistics since 2021, and CeMPA says "we have, as yet, failed to identify a suitable remedy." PolicyEngine's are older still, and carry no such flag.
Blocked on nothing. Needs no compute (#128) and no new harvest.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the scorecard's GAP_REASONS, divergence registry, and existing lane or parameter-comparison entry points. Review how calibration_relationship and source citations are represented, then record the certified PolicyEngine take-up parameters beside UKMOD's published assumptions. Done means the comparable aggregate rates, originating DWP/HMRC sources, dates, and mechanism caveats are represented without making value claims.
Written by the indexing model from the issue text.
Assessment
- Domain
- analytics, data
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 62/100