PolicyEngine / PolicyEngine/microcosm
Build K epic: transactional country executor via strangler extraction from the US release monolith
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 4
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 94
Description
Goal (architecture review, recommendation 5)
Replace the imperative US release monolith (tools/build_us_fiscal_refresh_release.py, ~6,600 lines) with a transactional country executor: CountrySpec compiles into a typed DAG of idempotent stages — intake → source/support construction → target materialization → selection/calibration → export → gates → release — where every stage reads and writes content-addressed artifact manifests and is independently restartable. Specs then govern the build (resolving #159's open question) instead of reviewing it.
Method: strangler, not rewrite — the monolith is the oracle
- Identity spine first (prerequisite, = review recs 2+3, extends #326): define
BuildRunIdentity(raw-input revisions/SHAs, facts+manifest SHA, country-spec fingerprint, donor hashes, selection identity-set SHA, seed + stage config, populace commit, PE-US/Core versions, canonical post-source frame hash) and the constraint-system fingerprint (CSR indptr/indices/data, target vector, stable record IDs, engine + materializer identity). Every stage keys its cache and its output manifest on these. This is design-sensitive (silently-wrong-if-wrong) — spec reviewed at Fable/Max tier before implementation. - Executor skeleton + toy conformance country: a minimal synthetic country runs build → gates → publish → immutable-load end-to-end in CI. This test is the platform's definition of done and every extraction must keep it green.
- Extract stages one at a time from the monolith behind the executor interface, newest-and-cleanest first (gates and export are already near-modular; target materialization has checkpointing; the solve gets its long-missing checkpoint here). Per-stage acceptance = byte-identical outputs vs the monolith on a pinned fixture run — the monolith stays authoritative until each stage's parity is proven, then that stage is deleted from it.
- Operational payoff is built-in: stages are naturally <30-minute processes with durable state between them — the jetsam discipline stops being a doctrine and becomes the architecture; a late-stage failure can no longer orphan a release manifest (each stage commits its manifest atomically; release marking is the final atomic step).
- Country migration order: US (it is the monolith) → UK (proves the abstraction on the second consumer; the ladder + local stack slot in as stages) → BE/NZ (should be nearly config-only if the abstraction is real). #366's release-delta report becomes trivial on staged artifacts (diff manifests stage-by-stage).
Sequencing vs live work
Build J's re-certification ships on the CURRENT monolith (the SSI fix does not wait for a re-platform). The identity spine (step 1) starts now — it hardens the monolith immediately and the executor inherits it. Steps 2-5 are the Build K campaign proper, running parallel to launch-tier paper/site work.
Division of labor
- Design (Fable + Max review): BuildRunIdentity + constraint-fingerprint field spec; the stage-DAG contract; per-stage acceptance criteria.
- Sol (gpt-5.6-sol, reasoning ultra) builds: per-stage extractions against the parity oracle, fingerprint implementation after spec sign-off, the toy conformance country — bounded, single-repo, loud-failure work.
- Claude agents: gate/certification semantics, release operations, cross-repo integration, review of every sol PR, issue/PR conventions.
- Human-only: this epic's go/no-go and its sequencing against the launch window.
Refs: the 2026-07-09 architecture review (codex-populace-architecture-review-20260709.md), #159, #326, #366, #368/#369 (gate machinery the executor absorbs), #305 (launch META).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tools/build_us_fiscal_refresh_release.py and the referenced architecture review, then read #159 and #326 for the identity-spine context. The first milestone is a reviewed BuildRunIdentity and constraint-fingerprint specification; eventual completion requires a toy conformance country in CI and byte-identical stage outputs against the monolith on a pinned fixture.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, build-system, data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100