PolicyEngine / PolicyEngine/microcosm

Doctrine: the scale ladder — dev-scale runs before production spend ("boring before billed")

Open
#624 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Chartered from the run 5–7 retrospective (#578). Total cost of the missing middle tier: ~$350 of Modal spend and ~5 days, with every major lesson (the PERIDNUM export bug, the 58-target silent skip, preemption/timeout economics) discoverable at 10% scale locally in under an hour.

The standing rules for every data pipeline in this repo:

  1. Scale ladder from day one. An identity-bound sample_fraction (uniform across survey spines, composition-preserving; donors always full) with three rungs: fixture (seconds — code paths), dev-scale real data run locally (sub-hour — mechanism + integration, including the export/publication tail), full scale (distributions + shipping). A pipeline without a sub-hour full-path rung is not chartered for cloud spend.
  2. Boring before billed. No full-scale dispatch until N consecutive dev-scale runs complete end-to-end clean — gates green or receipted. The first integration run's purpose is to reach the last phase; bugs concentrate in the least-exercised tail.
  3. Fixtures from real-data samples. Test fixtures preserve real pathology (nulls, dtypes, string encodings) — the week's misses were pathology-shaped, not logic-shaped.
  4. Gates at phase boundaries, not only terminal (the pre-simulation completeness gate is the pattern).
  5. Stated P(pass) before every gate, logged and resolved (_buildo-runtime/predictions/). Calibrated high dev-scale P(pass) is the productionization signal.

Antecedent: this is the encode-ladder discipline (fixture → single case → battery → full conformance) ported to data builds.

First consumer: the #578 stacked-spine adoption lane (single sample_fraction, 10% local builds, clone_attachment_fraction as a production-only memory lever).

🤖 Generated with Claude Code

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the run 5–7 retrospective in #578 and the stacked-spine adoption lane described here. Review the existing pipeline entry points and the _buildo-runtime/predictions/ records to determine where the scale ladder, phase gates, and pass-probability logging belong. Done means the doctrine is documented or adopted consistently, with the 10% local full-path run and its gates clearly verifiable.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.