PolicyEngine / PolicyEngine/microcosm

Epic: publish the US default from the graph-native build

Open
#956 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Tracks the path to publishing a certified US default built by the graph-native line, and the acceleration work along it. Opened 18 September 2026 from the data weekly's ask for an updated graph timeline.

Definition of done

tools/publish_release.sh runs on a certified US release built by the graph-native line; #893 is merged into main only after a from-scratch build, preflight and certification have run on its own tree (the rule in docs/us-release-build-rule.md, #947); main carries the same build as a standing check.

Where we are (18 September, evening)

  • The stacked line #893 ← #935 ← #945 ← #949 / #950 is merged level with main; every PR in it is mergeable. #951 (executor observer flag) and #947 (the rule and its four blockers) are merged.
  • Shared inputs: #952 (ASEC pins, one gate round from merge), #955 (labelled Chronicle feed re-pin, CI running), the SPM independence role as a build stage (six commits on us-spm-role-stage, lane in progress), fresh lineage accepted, PolicyEngine/chronicle#277 filed for the consumer-artifact refusal.
  • Measured: the 19-node graph completes at 1/1000 (2,028 CPU-s, 84% of it US-runtime source authentication outside the graph; a required replay with every node a store hit still costs 1,664 CPU-s). Nothing above 1/1000 has completed. The executor runs nodes sequentially. Modal is not wired for US runs.

Steps

Type: T = agent-bound, C = compute-bound, H = a decision.

  • 1. Byte transports lifted (lane on native-byte-transports, ACS budgets first) and the 1/15 run completes on its head — T then C, 1–2 days
  • 2. 1/10 run — C; skipped unless the 1/15 run meets a ceiling the census missed
  • 3. Stack merged downward through gates: byte transports → #949 → #945; #950 → #945; #945 → #935; #935 → #893 — T + H, 1–2 days
  • 4. Shared inputs on main: #952, #955, the SPM stage PR — T + H, 2–4 days
  • 5. #893 re-levelled on main so the graph build reads the labelled feed, the pinned ASEC inputs and the role stage — T, hours
  • 6. Full-source native run on #893's tree — C; 15–76 CPU-h on the sequential executor (1–3 days wall); the ledger gives a first attempt 25% and three attempts 70%, so plan for two: 3–6 days with fixes. Critical path.
  • 7. Preflight and certification on that output (release gates, reform-coverage smoke, SPM composition, target parity) — C + H, about a day plus decision 1 below
  • 8. #893 gate with the build receipt attached; merge — T + H, 1 day
  • 9. Standing from-scratch check on main — T to write, C to run
  • 10. Publish — H

Serial sum about 3–5 weeks. Current estimate (my ledger, 18 September): 10% by 1 October, 35% by 15 October, 60% by 1 November; median about 18 October. A Codex second opinion is planned for 19 September.

Acceleration, by leverage

  • A. Publish from main's tools first. When #952, #955 and the SPM stage land, run the base stage (46 min measured) and the release tool on the labelled feed and publish that certified release. The graph's launch criterion then becomes reproducing the certified default's receipts. Needs the operator's word for the build.
  • B. Parallel executor. Independent DAG branches execute concurrently behind a flag, with node keys, receipts and store objects byte-identical to the sequential run. Converts step 6 from days per attempt toward the longest chain. Lane brief written; PR against main.
  • C. Cut the 84%. Attribute the outside-graph source-authentication cost precisely and memoize what is content-addressed across nodes and runs. Same lane as B.
  • D. Skip the 1/10 run (step 2) unless the 1/15 run says otherwise.
  • E. A second machine or Modal for full runs, so fix attempts run concurrently instead of serially on the one 128 GiB box. Modal is not wired for US runs; 1–2 days.
  • F. Lane capacity: every reactivated Claude account is a lane; Codex returning gives Astra for the hard reviews.

Decisions

  1. The certification claim at step 7. A "certified release" is available on the --base-h5 arm now. Promotion as the default under #578's improvement gate needs the --exact-k arm, which needs a Chronicle consumer artifact (PolicyEngine/chronicle#277). Recommendation: publish as a certified release; let #578 promotion follow the Chronicle fix.
  2. Merge order for the stack (step 3): downward through gates as listed, or wait and merge #893 with everything at once after the build. Recommendation: downward now, so each piece carries its own review and the final gate reviews only the build receipt.

Not in scope

The UK graph work (María's calibration-on-graph PR, paused on #893) and the geography reordering (deferred until after the Autumn Budget, one-off test first) are tracked separately.

🤖 Generated with Claude Code

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with docs/us-release-build-rule.md and tools/publish_release.sh, then review the dependency chain through #893 and its listed inputs. The work is done when a certified US release from the graph-native line runs the publish tool, with preflight and certification completed on its own tree, and main carries the standing check.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, shell
Domain
build-system, release
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.