PolicyEngine / PolicyEngine/microcosm

Build K epic: transactional country executor via strangler extraction from the US release monolith

Open
#378 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Goal (architecture review, recommendation 5)

Replace the imperative US release monolith (tools/build_us_fiscal_refresh_release.py, ~6,600 lines) with a transactional country executor: CountrySpec compiles into a typed DAG of idempotent stages — intake → source/support construction → target materialization → selection/calibration → export → gates → release — where every stage reads and writes content-addressed artifact manifests and is independently restartable. Specs then govern the build (resolving #159's open question) instead of reviewing it.

Method: strangler, not rewrite — the monolith is the oracle

  1. Identity spine first (prerequisite, = review recs 2+3, extends #326): define BuildRunIdentity (raw-input revisions/SHAs, facts+manifest SHA, country-spec fingerprint, donor hashes, selection identity-set SHA, seed + stage config, populace commit, PE-US/Core versions, canonical post-source frame hash) and the constraint-system fingerprint (CSR indptr/indices/data, target vector, stable record IDs, engine + materializer identity). Every stage keys its cache and its output manifest on these. This is design-sensitive (silently-wrong-if-wrong) — spec reviewed at Fable/Max tier before implementation.
  2. Executor skeleton + toy conformance country: a minimal synthetic country runs build → gates → publish → immutable-load end-to-end in CI. This test is the platform's definition of done and every extraction must keep it green.
  3. Extract stages one at a time from the monolith behind the executor interface, newest-and-cleanest first (gates and export are already near-modular; target materialization has checkpointing; the solve gets its long-missing checkpoint here). Per-stage acceptance = byte-identical outputs vs the monolith on a pinned fixture run — the monolith stays authoritative until each stage's parity is proven, then that stage is deleted from it.
  4. Operational payoff is built-in: stages are naturally <30-minute processes with durable state between them — the jetsam discipline stops being a doctrine and becomes the architecture; a late-stage failure can no longer orphan a release manifest (each stage commits its manifest atomically; release marking is the final atomic step).
  5. Country migration order: US (it is the monolith) → UK (proves the abstraction on the second consumer; the ladder + local stack slot in as stages) → BE/NZ (should be nearly config-only if the abstraction is real). #366's release-delta report becomes trivial on staged artifacts (diff manifests stage-by-stage).

Sequencing vs live work

Build J's re-certification ships on the CURRENT monolith (the SSI fix does not wait for a re-platform). The identity spine (step 1) starts now — it hardens the monolith immediately and the executor inherits it. Steps 2-5 are the Build K campaign proper, running parallel to launch-tier paper/site work.

Division of labor

  • Design (Fable + Max review): BuildRunIdentity + constraint-fingerprint field spec; the stage-DAG contract; per-stage acceptance criteria.
  • Sol (gpt-5.6-sol, reasoning ultra) builds: per-stage extractions against the parity oracle, fingerprint implementation after spec sign-off, the toy conformance country — bounded, single-repo, loud-failure work.
  • Claude agents: gate/certification semantics, release operations, cross-repo integration, review of every sol PR, issue/PR conventions.
  • Human-only: this epic's go/no-go and its sequencing against the launch window.

Refs: the 2026-07-09 architecture review (codex-populace-architecture-review-20260709.md), #159, #326, #366, #368/#369 (gate machinery the executor absorbs), #305 (launch META).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tools/build_us_fiscal_refresh_release.py and the referenced architecture review, then read #159 and #326 for the identity-spine context. The first milestone is a reviewed BuildRunIdentity and constraint-fingerprint specification; eventual completion requires a toy conformance country in CI and byte-identical stage outputs against the monolith on a pinned fixture.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, build-system, data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.