PolicyEngine / PolicyEngine/microcosm
Column closure: every artifact column claimed by exactly one lineage class (spec totality)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 4
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 94
Description
Max, 2026-08-15: the process diagrams should have revealed the geography flaw sooner — the observability has to be total, not just cover the parts we extracted.
The measurement
Column-closure audit of the f025 artifact against the lineage spec (#695), 2026-08-15:
- 392 columns across 6 entities; spec claims 156 (+10 structural) ≈ 42%; 226 unclaimed.
- In the unclaimed dark matter: ~187 raw source (caps) columns; the 3 geography columns (where the #696 flaw hid); ~12
takes_up_*flags — stochastic draws with policy consequences, declared nowhere; capital-gains-tail donor receipts; spine-source ids and role/filing inputs.
Inventory: one JSON fixture of entity/column names extracted from the artifact (names only, no data).
The increment
- Total lineage classes in
specs/us_imputation_lineage.yaml:measured_native(per source channel),imputed(existing),computed(existing),assigned_by_ladder(#696 lane adds),take_up_draw(new — each flag's distribution, conditioning, parameter source, and calibration status),receipt,structural. - Column-closure conformance test: the artifact column inventory is a committed fixture; every column must be claimed by exactly one class entry; unclaimed or doubly-claimed ⇒ test failure. Inventory regenerated by a tool from any built artifact; drift between fixture and a fresh build fails loudly.
- Coverage meter on /lineage: claimed/total rendered in the hero, per class — the number that keeps us honest.
- Immediate deep-dive owed: the
take_up_*mechanism (where drawn, from what) documented into the spec as part of this increment — it is currently exactly as invisible as geography was.
Sequencing: lands after the two wave-1 lanes (#696 block-first, per-family predictor sets) merge, since all three edit the same spec file. Owner runs it as its own PR.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with specs/us_imputation_lineage.yaml and the committed artifact column inventory, then locate the tool that regenerates the inventory from a built artifact and the /lineage rendering. Run the column-closure conformance test first. Done means every artifact column has exactly one lineage class, fixture/build drift fails, take_up_* behavior is documented, and coverage is shown per class.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, yaml
- Domain
- data-engineering, observability, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100