PolicyEngine / PolicyEngine/microcosm-dynamics

Node graph implications for dynamics: registry, certification scopes, panel keying, and two interface amendments

Open
#412 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1
Forks
1
Avg merge
1h 46m
Merged PRs (30d)
28

Description

Microcosm's node graph merged into main today (PolicyEngine/microcosm#836; charter in docs/graph-acceptance.md, 37/37 properties green including parity with the unchanged UK spine and the US post-transfer slice). This issue records what that changes for the dynamics design: what this repo no longer needs to build, what the graph forces us to decide, and two interface amendments the Dynamics operator needs before the panel schema can land.

Mechanism claims below are from the merged code, not the plan: a node declares the slices it reads, the cells it owns, its params, and a kernel; its key is a hash of the normative declaration, its input artifact keys, and the kernel's implementation hash; structural nodes (CREATE, FILTER, EXPAND, REWEIGHT) open population versions; gates are kernels with the gate role and a release tier is certified only if every gate on the ancestral path passed; decisions live in the run manifest and never in keys; seeds derive from node keys; artifacts live in a content-addressed store.

What this repo no longer needs to build

  • Component registry and immutable candidate specs (#106 findings 1, 2, 3, 5). A candidate is a Graph. ComponentRef(kind, implementation_id, params) is a Node with a kernel ref, params, and the kernel's implementation hash. A candidate composed of c1, c5, c6, c8 and c9 is a graph of those nodes, and the store reuses every artifact whose key is unchanged. "Unchanged" means the same key, never the same code object. The engineering replay the review asked for (port the c14 lineage, compare against the committed rate cube, record a compatibility certificate) is a parity property of the same kind as microcosm's H1 to H3.
  • Certification scopes per tranche (#106 finding 4; #113's "inherit only the certification scopes their slice uses"). A cell is certified only if every gate on its ancestral path passed, so a product that prunes rows, columns, or years inherits exactly the gates its remaining cells descend from. No separate certification_scope machinery.
  • Pre-registration mechanics. gates.yaml thresholds become gate-node params inside the frozen declaration: an amendment changes keys and is visible as a new revision. Decisions that may not affect keys live in the manifest, and a release that needs an unmade decision stays unreached.
  • Seeds and the mean-over-N-draws protocol. Seeds derive from node keys, so a draw index is a param and every draw is reproducible and addressable.
  • The alignment layer (#113 hard part 2). Each projected year's alignment to Trustees assumptions is a REWEIGHT node that owns that year's weights and whose key includes the targets.

What the graph forces us to decide

  • Keying. microcosm's DESIGN.md defers (person_id, period) to "before the Dynamics operator". In graph terms the choice is a person_period entity produced by an EXPAND node with lineage to the person, versus a composite key on the person entity. Recommendation: the EXPAND form. It reuses lineage, membership remapping, and the row-mask primitive (Slice.rows names a boolean column), so a year-slice is a row mask on the period column, the same move as geography.
  • Weight-share. Households recompose across periods, so period-t group weights must be a declared function of member trajectory weights. That is a REWEIGHT node owning the period household's weights. Ownership makes the operator's absence a compile error rather than a silent member-constant collapse.
  • Repo boundary. Kernels (transition models, generators, gates, the transport gate) live here; the population model and structural deltas live in microcosm-graph.

Two interface amendments the Dynamics operator needs (microcosm side)

  1. Entry without lineage. EXPAND requires every new row to name an existing source id (population.py raises on unknown lineage). Births can trace to a parent; immigrant entry cohorts cannot enter at all. Proposal: either a union with a CREATE'd entry cohort, or an EXPAND form that admits sourceless rows carrying entry markers (birth, immigration), with exit markers as the FILTER side. decl.py is frozen, so this is an amendment ceremony.
  2. Period residency in mass accounting. The mass ledger and the design-weight anchor are per population version. "A trajectory's weight contributes to a period only while resident" needs both to be period-aware.

Numeric claims

The H2 parity work found that the FRS root transform's household weights differ by one ulp between arm64 and x86 for two of 135 fixture households, and forest kernels are declared tolerance-bound. Gate verdicts here must compare in-process or against declared tolerances; the half-versus-half noise floor in gates.yaml is already the right shape.

Sequencing

Migrate the registry and candidate ladder onto graph declarations before M5, otherwise the #106 P1 refactor builds a second registry. Land the panel schema with M5 as #113 planned, but open the two amendments first. The transport gate itself is a gate node whose inputs are the PSID-fitted artifact and its deployment on the CPS frame.

Reconcile with in-flight design

  • #121 already ported the family-transitions registry with a c16 compatibility certificate. The proposal here is that the next step expresses those components as graph nodes rather than growing that registry further.
  • #218 (immigration: new-entrant cohorts through the scheduled-entries seam) is the design the entrants amendment must serve; whichever form the amendment takes, the scheduled-entries seam should be its consumer.
  • #217 (alignment machinery with displacement disclosure) maps onto per-year REWEIGHT nodes whose keys carry the targets; the displacement disclosure is that node's receipt.

Refs: #106, #113, #121, #217, #218, PolicyEngine/microcosm#836.

🤖 Generated with Claude Code

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with docs/graph-acceptance.md, then inspect population.py and decl.py for the lineage and declaration constraints described here. Review #106, #113, #121, #217, and #218 to understand the in-flight designs. Done means resolving the entry-lineage and period-aware mass-accounting amendments and sequencing the graph-based migration before M5.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.