PolicyEngine / PolicyEngine/microcosm
WS-E spine deferrals: known bugs and signed inconsistencies awaiting their increment
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 4
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 94
Description
Parent: #145 (workstream E of the UK-migration epic #665). The spine-lane counterpart of #736 (WS-C calibration deferrals): a central checklist of every deferral, known bug, and signed inconsistency filed while the spine lane landed (#686 → #757 → PR #793), so nothing lives only in a PR body or an adjudication comment. Items graduate to their own issues when they become actionable; the committed signed-differences register (uk/spine_swap_signed_differences.json) remains the authoritative record of every intentional candidate-vs-incumbent deviation.
The capstone: the first certified cut
-
Re-certify the candidate-of-record on a post-#755 spine. #750 landed via #755 (2026-08-28), so spine-i no longer carries the UC lever the dispositions assume;
spine-j(data/ukds/acceptance/spine-j-remeasure/, developer full build from main, gates green,private_pension_wealthpresent) needs the RC-strict battery + ladder + parity instruments + signed-differences re-mint before it can replace spine-i as candidate-of-record. Precedes the cut. -
Cut the first certified
microcosm_uk_2024line. The queue is down to 6 cells, all UC-side (v12, spine-j + the #810 register revision: the 4 #792 artifacts cleared with #803, the sparse-band siblings cleared with the #807 lift, CGT slid inside the band): 2 TCL disability cells (#808) + 3 composition cells and 1 payment band (#809) — diagnosed as measurement-grain defects (see the 2026-08-28 capstone comments), re-bind PR designed. Also owed before the cut: the RC-strict battery + parity instruments + signed-differences re-mint on a post-#755 spine (spine-j is a developer build; spine-i's 14/14 battery does not transfer). Blocked-run receipts:data/ukds/acceptance/757-swap/(13-cell era),runs/uk-623-first-calibrated/spine-assessment-v12/(current). (2026-08-28 state: the queue is 8 cells on spine-j — the 4 #792 artifacts cleared with #803, and #750, now landed and measured (+1.11m UC support, receiptsruns/uk-623-first-calibrated/spine-assessment-v11/), moved UC caseload inside/near the bound; the 8 are dispositioned across #807 (sibling distortion), #808 (2 TCL disability cells), #809 (3–4 UC composition cells + the newly failing COUPLE_WITH_CHILDREN payment band), andobr.capital_gains_taxat exactly −25.0%, an adjudication candidate.) -
B4 completion / C4 (#757 items 9, 14): flip
uk_input_mass_parity's reference to the just-cut line with run provenance, then retire theefrs-post-calibrationdescriptor and the incumbent filename pin (weighted_integrity.py,microcosm-data/contract.py). Cannot precede the cut (US #327 doctrine). The frozen eFRS parity extraction stays as the historical instrument. -
W8 (#757 item 12): certified-identity registry entry for the cut artifact.
-
#731 scorecard probe + rule-1 scorer seal: run against the certified line; the certification producer already cross-pins the score receipt.
-
UK_REFERENCE_DATASET_NAMEre-semantics once a second certified line exists — "reference" becomes the previous certified microcosm line (#757 item 13 note,terminal_gates.py). -
Retire the five obsolete release-fit deferrals: UC one-child, two-child, five-plus-child and lone-parent counts, plus the HMRC private-pension count £100k–£150k. Completed in #892; all five remain in calibration and return to ordinary enforcement under the unchanged 25% bound. The evidence uses 1,500 updates /
family_equal; this does not complete the certified cut.
Filed spine issues
-
#785 — derive NHS allocation (and audit age-conditioned imputations) from the disaggregated age surface; was blocked on the swap certifying, now unblocked by #787's merge
-
#791 — relationship-to-head frame column so the ONS household-composition split is measurable; its three windowed exclusions expire 2026-11-26
-
#750 — split
private_pension_wealthout ofcorporate_wealth(uk-data#452 M2): landed via #755 + policyengine-uk#1824 (2.92.0) + floors #811; lever measured (UC support +1.11m, caseload −25.6% → −20.8%); residual owned by #809/#808 — landed via #755 and first measured 2026-08-28: UC initial support 3,128,938 → 4,242,978 with non-UC targets untouched; the residual is #809 -
#725 — CGT top-tail age and geography joints vs HMRC marginals
-
#721 — carry manifest rewrites into the shared Stage/StageRecord producer graph (in-place rewrites make stage-input state unreconstructible; E5's out-of-build refit instrument binds only pre-rebase)
-
#713 — tighten the UK input-mass tolerance by receipting the explained top drifts (the gate note declares the re-mint procedure)
-
#688 — the open half of the spine-grain adjudication (E10 gate arithmetic)
-
#685 — net-new UK stages: UC deduction attributes, bus fares, WAS debt
-
#687 — legacy 2023-vintage spine build (deferred)
-
policyengine-uk-data#469 — the incumbent's ETB per-head division (upstream defect of the #467 family, observed during E10)
-
#807 — re-measure the hmrc SPI band exclusions: three "sparse cell" premises fail on spine-i and spine-j (77–1,158 in-band carriers); lifting them re-couples the amount/count pairs (the +38.6%
self_employment_income_count_50k_70ksibling distortion is the live symptom) -
#808 — child disability under-identification on the FRS disability stage: the two TCL disability cells run −56.5% / −40.0%, invariant across spine-c/i/j; blocks 2 of the 8 capstone cells
-
#809 — the UC residual after the pension-wealth split (support 4.24m vs 6.76m): promote the wealth-block re-draw undershoot from #750's follow-ups, re-measure the U8
would_claim_uc0.55 freeze on its requested new evidence, and the composition shape (2-children fails while 1 and 3 fit); owns 3–4 of the 8 capstone cells -
#828 — preserve FRS benefit-unit capital for UC while retaining WAS household wealth: carry
TOTCAPB4intouc_reported_capital, preserve the WAS wealth surface and top tail, make SPI UC/capital writes coherent, and refreshwould_claim_ucafter the final reported-UC write; no disagreement-conditioned twins. -
#832 — make SPI UC receipt coherent with benefit-unit composition and combined income, and add a UC-specific child count for the DWP numbered-child targets; successor to #828 after its capital-coherence fix.
-
#834 — port the TFC work recently done in uk-data into microcosm.
-
#840 — improving SPI income imputation to preserve household characteristics like family structure or employment type
-
#862 — FRS stages evaluate policy parameters at the survey period (uk-data#475 disability-threshold year and uk-data#478 DSA gate, inherited via parity; uk-data#476 reduces to the SPI-refresh week mix; uk-data#477 is covered by
frs_release.json+test_uk_frs_release; uk-data#479 is an input to #790; incumbent fix uk-data#480) -
#866 — savings interest income: the spine carries £13.8bn at 2025 vs HMRC SPI £18.3bn (taxpayers, 2023-24) vs the ONS D.41 row £95.5bn; a frame gap plus a target-concept gap (measure exclusion
ons.savings_interest_income, #762 A16, expires 2026-10-03) -
#867 — housing benefit £2.7bn vs OBR £12.1bn: pensioner / supported-accommodation HB after the UC migration (exclusion
obr.housing_benefit, A16, expires 2026-10-03) -
#868 — SLC plan-2 borrower stocks at a third of SLC's: the FRS flag captures repayers, not the liable stock (exclusions
slc.borrowers.plan_2_liable,slc.borrowers.plan_2_above_threshold, A16, expire 2026-10-03) -
#869 — JSA claimants 2.8× the DWP caseload: legacy-benefit reporting after the UC migration (exclusion
dwp.jsa_claimants, A16, expires 2026-10-03) -
#930 — UK bus fares: impute journeys from the NTS microdata (UKDS SN 5340) priced at the published revenue per journey, so the DfT receipts targets bound in #904 are no longer level-set by the #890 lcfs rake and incidence draw (María's ruling 2026-09-15 on #927; rake and target kept together as the interim, ONS 07.3.2 recorded as a diagnostic)
Deferrals and signed inconsistencies without their own issue yet
- #750 residuals (graduated 2026-08-28 on closure): the composite mass audit (old corporate wealth vs corporate + pension, so the input-mass gate distinguishes the intended transfer from pension-mass loss); the cash-ISA gap (
cash_isais capital for the means tests but the model'ssavingsmaps savings accounts only);current_db_pension_wealth(£1,149bn current-employment DB not carried). The mirror→re-pin ordering stays on B4 above; the re-draw undershoot is #809 M1 + #145. -
macro_realismfor the UK (#757 item 9): named in the arming item but never declared — nouk/gates.jsonentry and noUK_GATE_REGISTRYbinding exist (only the Belgium precedent). Decide arm-or-drop; arming needs a UK metric/band source plus a new binding. - Household-surface QRF tail gate: the 12
household_qrf_tailgrids (was_wealth surface) are measured in the #686 L3 baselines but not armed —uk_qrf_tail_concentrationcovers the 47-column HMRC person surface only (deferred in PR #793; needs a household-surface evidence function and gate entry). -
uk_take_up_signalsemantics: the gate sits terminal, so it compares weighted post-calibration shares against unweighted contract targets — not the same statistic pre/post calibration. Flagged during #787; unruled. -
owned_landseed instability (its reviewed input-mass exclusion expires 2026-09-26): the winsorised-donor / separate-land-imputation revisit registered on #145 (comment 5341180811); the reviewed input-mass exclusion was re-signed 2026-08-26 on a fresh 25-stage stability receipt (swing persists at 53.8% national) and expires 2026-09-26 — re-sign on fresh evidence or fix before then. - E8 mass-change reasons: five pre-E8 stage families still declare the generic fallback reason instead of a stage-named one (standing follow-up from the #740 adversarial round 2).
- E7 reviewed-absent incapacity:
hmrc_spi_incapacity_benefit_incomeis reviewed-absent per the #683 adjudication; the stage fence (not the gate) promotes it when a future FRS vintage carries signal. Standing, self-enforcing — listed so the convention is findable. -
communication_consumptionregister scope: the one LCFS column where the incumbent is closer on level while the candidate is closer on share; signed inside the donor-faithful class, with a one-line rescope on offer if the adjudicator objects (#686 ledger, still open). -
student_loan_balancedirection: signed on the register with the direction unevidenced beyond the E5 donor-truth measurement (#686 ledger, still open). - #609 disclosure confirmation for the published parity numbers (carried from the #686 queue).
- T4 lead — LCWRA element caseload (both artifacts ~3x the anchor): the T6-era admin battery (first run 2026-08-30, receipts
uk-candidate-eval/out/v15/t4_admin.json) measures UC LCWRA-entitled at 7.42m (candidate) and 6.48m (incumbent) against the DWP Stat-Xplore anchor of 2.42m (LCWRA-Higher entitlement, Apr-Dec 2025 mean, chronicle#187/#188). Both sides are wildly over, so adjudicate the probe's semantics (entitlement flag vs the paid element) before treating it as a spine defect — but the candidate runs 0.94m above the incumbent either way. - T4 lead — multi-benunit household formation (candidate 2.3x the incumbent): the multi-family-household differentiator reads 6.57m (candidate) vs 2.84m (incumbent) vs the ONS Families and Households anchor of 244k. The anchor semantics need adjudication (the probe counts multi-benunit households; ONS counts multi-family), yet the candidate-incumbent gap is anchor-independent: the spine forms far more multi-benunit households than the incumbent, and the candidate lost all 4 T4 differentiators. Household/benunit formation shape is a spine question.
- T4 lead — income-tax mass shortfall: income_tax_total measures GBP 280.6bn (candidate) vs GBP 293.3bn (incumbent) against OBR EFO March 2026 Table 3.4's GBP 331.4bn — the candidate sits 3.8pp further under than the incumbent (-15.3% vs -11.5%) despite winning the SPI band surface 110-24 on the frozen register. The mass sits somewhere the banded targets do not bind; candidates: top-tail amounts, non-SPI channels, the calendar-average convention.
- Local-area support floor — the two micro-authorities (#762 A4 / A14, 2026-09-02): Isles of Scilly (E06000053) and City of London (E09000001) cannot reach the 50-row / ESS-50 / 50-source floor by clone count (7 and 14 positive-weight rows at K=4; 17 and 49 at K=10 on spine-m); reviewed exclusions in
uk/local_area_support_exclusions.json, expiring 2026-12-02, re-measured on every candidate (a passing area retires the entry); their own local-authority cells are signed-deferred (local_authority_support_floor_excluded, 43 cells). Spine-side because the fix is frame rows for those areas, not targets. No issue yet. - Council-tax band H spine support (#762 A14, 2026-09-02; re-measured on microcosm#934, 2026-09-15): spine-r carries the same 49 raw FRS band-H households, so the 296 English authority band-H cells stay signed-deferred (
council_tax_band_h_spine_support_absent, no expiry) and Shetland's band-H cell joined them (no band-H clone at K=15). The 53 Welsh and Scottish band-H cells that do bind carry the family's three largest misses (Merthyr Tydfil and Na h-Eileanan Siar with targets of 2 and 5 dwellings, Stirling −43 %). Spine-side: more band-H donors or a band-H-aware draw retire it. No issue yet. - Council-tax band support beyond band H — per-clone band redraw (microcosm#929 CT-B, graduated 2026-09-16 on closure):
council_tax_bandis the raw FRSctbandcopied verbatim to all 15 clones, so an authority's band distribution can only be approached by reweighting rows that carry some other authority's band. After the taxbase re-base (microcosm#934) the family fits 99.6 % of its 2,511 authority cells within 10 %, and the residue is the band column: Na h-Eileanan Siar band A (−31 %), Derbyshire Dales band G (−38 %, the same cell on the VOA basis) and the three band-H micro-cells above. The route is a per-clone, rank-preserving redraw ofcouncil_tax_bandfrom the assigned authority's taxbase band distribution (theassign_brma_by_cell+stable_identity_uniformspattern), withcouncil_taxre-imputed per clone; it is also the only route to lifting the band-H deferral at K=15. Spine-side. No issue yet. - Per-block engine resolution mis-measures population-normalised formulas (#762 R15, 2026-09-03):
--engine-blocks Kin the rowwise candidate driver multipliescorporate_land_value(andland_value) by K because policyengine-uk allocates a fixed national aggregate by share of the block's total weighted corporate wealth; the release posture is single-block and the manifest now names the affected measures, but the driver flag still exists for constrained machines. Tooling-side, recorded here because it produced two false spine "defects". No issue yet. - Cross-lane markdown sweep (#757 item 16 remainder):
COVERAGE_PROGRESS.md,PROGRESS_BUILD*.md,_698-*.md,_*LANE-NOTES.md,FINAL_REPORT.md— historicize or delete in their own PR; the UK-lane markdowns were dispositioned in #787.
Standing records (not action items)
- The U8
would_claim_uclever is measured and re-frozen at 0.55: the raise moves claimants off legacy benefits (housing benefit −0.0% → −52.4%) while leaving every UC cell unchanged; the contract entry carries the receipts. Do not re-raise without new evidence — the lever's disposition is #750. - The spine signed-differences register (14 entries incl. the quantitative bounds re-minted per candidate) is the swap-decision record; per-stage battery gates are the standing health checks.
Refs: #686, #757, PR #793, #736 (calibration counterpart), #665.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the unchecked entries in this checklist, especially the B4 completion item and the signed-differences register at uk/spine_swap_signed_differences.json. Inspect uk/gates.json and the UK_GATE_REGISTRY binding for the macro_realism item, then follow the referenced issues and receipts. Done means each actionable deferral has its own issue or a recorded disposition, with the authoritative register and remaining gate work updated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100