PolicyEngine / PolicyEngine/microcosm

Promotion for the constant-id national line: pointer and certified-loader extension

Open
#823 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

#806 / #812 delivered the UK national line's inspect lane: assemble a certified cut, publish with --no-latest under an immutable per-cut tag (microcosm-uk-2024-25-national-<YYYYMMDDTHHMMSSZ>-<uuid8>), and adjudicate it on the dashboard via the release list / ?release=. Promotion — moving latest.json — is structurally refused there (publish_release rejects a pointer move for any tag that is not the release id itself), because the current pointer and loader contracts cannot express it. This issue is the design and implementation that lifts that refusal.

Why promotion cannot work today

The 2026-08-27 ruling makes the national release id constant across cuts; run identity lives in the per-cut tags and the Logbook chain. Two contracts still assume id == tag:

  1. The pointer cannot name a cut. latest_pointer_payload carries only {schema_version, release_id, updated_at, paths} (release.py). A promoted pointer could say the current release is microcosm-uk-2024-25-national but not which cut that means.
  2. The certified loader follows the id as a tag. loader.py fetches release_manifest.json at revision=pointer.release_id — an immutable tag named exactly the release id, which the national line never mints (and must not: minted once at cut 1 it would pin cut 1 forever) — and separately hard-asserts the default artifact's revision == release_id, which per-cut revisions fail by design.

The dashboard itself would survive a pointer move (it reads releases/<id>/ paths off main, which always hold the current cut); every certified consumer would break.

Design decisions this issue owns

  • Pointer shape: add a tag/revision field naming the current cut (bumps LATEST_POINTER_SCHEMA_VERSION; the reader refuses unknown versions, so microcosm-data consumers, the dashboard, and the pe certification path upgrade in lockstep) — versus teaching the loader to read the manifest from main and trust its per-cut revision + sha pins (no schema bump, different immutability trust model).
  • Repo pointer semantics: latest.json is repo-global, and policyengine/populace-uk-private currently serves the June 2023 release to certified consumers. Promoting the national line re-points what "current" means for the whole repo: one shared pointer, per-line pointers, or separate repos is an adjudication, not a refactor.
  • Registry entry for the national line (the registry today has only (uk, 2023) → populace_uk_2023.h5), so pe can load what the pointer names.
  • Lift the interim refusal: replace publish_release's per-cut-tag pointer refusal with whatever the new contract permits, keeping the fail-closed default for tags the pointer cannot express.

Riding along

  • The constant-named staging branch release-staging/microcosm-uk-2024-25-national strands when tag creation 409s mid-publish and blocks the retry until deleted by hand; a per-attempt staging branch name removes the recurrence.

Acceptance

  • A promoted national cut resolves end-to-end: pointer → current cut → certified loader fetches the manifest and artifacts at an immutable ref and verifies their digests.
  • Certified consumers of the 2023 line keep working through the transition, per the repo-semantics ruling.
  • The inspect lane is unchanged: --no-latest publishes still work identically, and promotion of a tag the pointer cannot express still refuses.

Out of scope

  • The uk_target_fit adjudication line (#750 et al.) — a shippable certification is a precondition for having anything to promote.
  • Dashboard feature work; it already reads current cuts from main.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading release.py's latest pointer and publish_release contracts, then trace loader.py and the registry entry used by certified consumers. Decide the pointer and repo semantics, implement the compatible promotion path, and verify that promoted cuts resolve through immutable refs while 2023 consumers, --no-latest publishing, and refusal of unsupported tags continue to work.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devops, release
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.