PolicyEngine / PolicyEngine/microcosm

Sweep declared manifest outputs against written columns before they become load-bearing

Open
#700 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Follow-up from the PR #690 review (workstream E of #665; see also #145).

The class

The licensed-data acceptance on #690 caught a declared manifest output (state_pension) that was an SPI concept, not a persisted frame column — dormant for as long as outputs were purely declarative, refused at full rung the moment country_stage_plan compiled outputs into StagePlan.produces. As the review put it: the other manifests may carry the same latent class, dormant until they too become load-bearing. Each discovery via a build costs a credentialed full-rung run; a static sweep costs one pass.

Scope

Validate declared outputs against what the runtimes actually write, statically, for every source manifest not yet load-bearing:

  • us/source_stages.json (39 stages) against the us_plan stage produces declarations and runtime constants — note the US plan has stages with no manifest entry, so the mapping needs care rather than a naive zip.
  • be/source_stages.json against its implementations.
  • uk/cgt_source_stages.json (the bespoke CGT declaration from #676) against cgt_imputation's written columns — also a candidate for absorption into the canonical uk/source_stages.json, which would make its outputs load-bearing the same way.
  • The frozen uk/hmrc_income_source_stages.json is a historical evidence anchor with one known latent entry (the state_pension concept) — document, don't edit; it retires at the FRS 2024-25 refresh (#687).

PR #690 already pins the two migrated UK stages' outputs to named runtime written-column constants (TestDeclaredOutputsAreWrittenColumns); this issue extends the discipline to the rest.

Acceptance

A conformance test (or one per country package) that fails when a declared output names a column no runtime constant or plan declaration backs; every existing latent entry either fixed or documented with an owner.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the existing TestDeclaredOutputsAreWrittenColumns coverage from PR #690, then inspect us/source_stages.json, be/source_stages.json, uk/cgt_source_stages.json, the us_plan declarations, and cgt_imputation's written-column constants. Account for US stages without manifest entries and keep uk/hmrc_income_source_stages.json unchanged. Done means conformance coverage catches unsupported outputs and every latent entry is fixed or documented with an owner.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.