PolicyEngine / PolicyEngine/microcosm
Evidence-tier publishing: ship the best available artifact at all times, with its failures recorded in the manifest (dense first)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 4
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 94
Description
The gap (owner question, 2026-07-22): "shouldn't we publish the best version we have available at all times?"
Today there are exactly two publication states: certified default (all gates green, latest.json, pe.py manifest) and nothing. Build N dense completed the first full dense battery since Build J — the best full-pool national artifact that has ever existed — and it lives only on a local disk, because rc=1 means no release manifest and the publish contract requires one. Zero dense tags exist on policyengine/populace-us (20 tags checked 2026-07-22; the nearest artifact is buildl-acs-local, a different product — local-area ACS expansion).
The owner's instinct is right: best-available should be publishable at all times. The wrong implementation would be bypassing gates (that spends the meaning of "published" — the #492 doctrine and today's entire register-alignment arc exist to prevent exactly that). The right implementation is a declared evidence tier, generalizing the #490 pattern (adjudicate, record, ship with the caveat carried in the artifact):
- An evidence-release contract in populace-data (sibling of
validate_release_dir, not a relaxation of it): requires the same required files PLUS a mandatoryknown_failuresblock in the release manifest — every gate failure the run recorded, verbatim, with owners (e.g. dense: the Table 1.4 CG row at −30.2% under the 5× stretch bound, owner #487; the 7-column dense tail set with measured shares, owner #481/#487). An evidence release with an EMPTY known_failures block is invalid — the tier exists to carry failures honestly, not to hide them. - Builder support:
--evidence-releasewrites the manifest with the known_failures block populated from the actual gate results instead of refusing to export. NOT a bypass flag on the certified path: the certified path is untouched; this is a different output contract, and the artifact name carries the tier (…-evidence-…). - Publish mechanics: immutable tag as usual, NEVER latest.json, never the pe.py bundle. A separate
latest-evidence.jsonpointer is allowed so consumers can discover the best current evidence artifact. - Supersession: "best available at all times" means best CURRENT — each new evidence publish supersedes the last (Build O dense, running now, would be the first published artifact of this tier rather than N dense).
Non-goals: no change to certified gating; no gate-bypass flags on the certified path; evidence artifacts never feed pe.py certification.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the populace-data validate_release_dir contract and the builder's certified export path described in the issue. Trace how actual gate results become the release manifest, then define the separate evidence-release contract and publication pointers without changing certified outputs. Done means evidence artifacts carry non-empty known_failures, use immutable tags and latest-evidence.json, and never update latest.json or the pe.py bundle.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, release
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100