PolicyEngine / PolicyEngine/microcosm

Evidence-tier publishing: ship the best available artifact at all times, with its failures recorded in the manifest (dense first)

Open
#506 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

The gap (owner question, 2026-07-22): "shouldn't we publish the best version we have available at all times?"

Today there are exactly two publication states: certified default (all gates green, latest.json, pe.py manifest) and nothing. Build N dense completed the first full dense battery since Build J — the best full-pool national artifact that has ever existed — and it lives only on a local disk, because rc=1 means no release manifest and the publish contract requires one. Zero dense tags exist on policyengine/populace-us (20 tags checked 2026-07-22; the nearest artifact is buildl-acs-local, a different product — local-area ACS expansion).

The owner's instinct is right: best-available should be publishable at all times. The wrong implementation would be bypassing gates (that spends the meaning of "published" — the #492 doctrine and today's entire register-alignment arc exist to prevent exactly that). The right implementation is a declared evidence tier, generalizing the #490 pattern (adjudicate, record, ship with the caveat carried in the artifact):

  1. An evidence-release contract in populace-data (sibling of validate_release_dir, not a relaxation of it): requires the same required files PLUS a mandatory known_failures block in the release manifest — every gate failure the run recorded, verbatim, with owners (e.g. dense: the Table 1.4 CG row at −30.2% under the 5× stretch bound, owner #487; the 7-column dense tail set with measured shares, owner #481/#487). An evidence release with an EMPTY known_failures block is invalid — the tier exists to carry failures honestly, not to hide them.
  2. Builder support: --evidence-release writes the manifest with the known_failures block populated from the actual gate results instead of refusing to export. NOT a bypass flag on the certified path: the certified path is untouched; this is a different output contract, and the artifact name carries the tier (…-evidence-…).
  3. Publish mechanics: immutable tag as usual, NEVER latest.json, never the pe.py bundle. A separate latest-evidence.json pointer is allowed so consumers can discover the best current evidence artifact.
  4. Supersession: "best available at all times" means best CURRENT — each new evidence publish supersedes the last (Build O dense, running now, would be the first published artifact of this tier rather than N dense).

Non-goals: no change to certified gating; no gate-bypass flags on the certified path; evidence artifacts never feed pe.py certification.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the populace-data validate_release_dir contract and the builder's certified export path described in the issue. Trace how actual gate results become the release manifest, then define the separate evidence-release contract and publication pointers without changing certified outputs. Done means evidence artifacts carry non-empty known_failures, use immutable tags and latest-evidence.json, and never update latest.json or the pe.py bundle.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, release
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.