Optimize preview builds: phased plan for the fastest PR previews (tracking)

Summary

Umbrella tracker for making PR preview builds fast across the lecture repos, from a measured, adversarially-verified July 2026 analysis. Mechanisms, savings arithmetic, rejected alternatives (single-page builds, Sphinx -j, CPU migration of the GPU repo, conda-cache-instead-of-containers, and more) and open questions live in PROJECT-OPTIMIZE-PREVIEWS.md; this checklist, not the doc, is the status tracker.

Headline finding: preview time goes to fixed overhead, not to building the author's change. The 40-minute cold builds were a silent cache-freshness failure (#83), not inherent cost — speed work and the #83 alerting fix are one project.

Measured baseline

• lecture-dp PR, warm cache — ~2.7 min: container pull 129s (75% of the run), HTML build 22s, cache restore 3s.
• lecture-dp PR, cold cache — ~40 min: notebook re-execution 2,350s; hit every PR for ~2 months while the weekly cache builder failed silently (#83).
• lecture-python.myst PR, every time — ~17 min: PDF 379s + notebooks 117s (formats unused in previews), conda 102s + JAX 55s (bakeable into the custom AMI), checkout 101s, HTML 80s, deploy 61s.
• Lean image pull: 2,741 MB compressed, 1,233 MB of it TeXLive — unused by an HTML-only preview.

Expected results (today → Phase 1 → end state)

• lecture-dp small edit: ~2.7 min → ~2.2–2.5 min → ~1.5–2.5 min incl. deploy, links at ~20s.
• lecture-dp cold miss: ~40 min silent → ~10× rarer, loud, once per PR → ~18–28 min, rare.
• lecture-python.myst PR: ~17 min → ~8–9 min → ~3.5–5 min (prose ~2.5–3.5).
• Second push, same PR: same as first → execution skipped when code unchanged → ~1.5–2 / ~3.5–5 min.

Caveat on the lecture-dp rows: its ci.yml has no preview deploy step today — it builds HTML and uploads an artifact — so "incl. deploy" is for a preview still to be stood up.

Phase 1 — days, all S-effort, independently shippable

Critical path is the first item: it alone outweighs every actions-repo item here combined, and is the only one gated on a decision (Decision 1) rather than effort.

• lecture-python.myst: HTML-only PR previews; three-format build behind a full-preview label; PDF/notebook coverage moves to weekly cache.yml (−7–7.8 min/PR). Not started — .github/workflows/ci.yml still runs the notebook build (--custom-builder=jupyter, :72) and pdflatex (:84) on every PR, ungated. → #148 (gated on #146)
• #83 alerting fix: curl REST fallback + ::error:: guard in create-failure-issue.sh; upload-failure-reports: true inside build-jupyter-cache — done in v0.11.0 (#122, #127) by another mechanism: build-jupyter-cache/scripts/create-failure-issue.sh was deleted (a5da9f3) rather than hardened, and filing moved to actions/github-script (build-jupyter-cache/action.yml:385), sidestepping the github.action_path/no-gh/label-validation trio that made the old path fail silently in container jobs. upload-failure-reports shipped as an input defaulting true (build-jupyter-cache/action.yml:42-51), passed to all three inner build-lectures calls (:168, :178, :188). #127 covered the abort-before-builds case.
• lecture-dp: save-cache: 'true' on the restore step (one line). Its ci.yml still calls quantecon/actions/restore-jupyter-cache@v0 without it. The input exists since v0.5.2 (#24), the save-mode outputs/fail-on-miss bugs were fixed by #104 in v0.9.0, and the path is CI-covered since v0.10.0 (.github/workflows/test-actions.yml:270, :395) — nothing blocks it. → #149
• lecture-dp: cache-on-merge job on push to main (paths-ignore: [environment.yml]; weekly rebuild stays the anchor; no actions-repo change). Not started — its cache.yml triggers on the weekly cron plus push: paths: [environment.yml], the inverse of what this needs. → #149
• --depth=1 on the two still-unbounded SHA fetches in scripts/detect-changed-lectures.sh:26-27 — this repo, shippable today. lecture-python.myst has no copy of the script (it calls preview-netlify@v0, which invokes ours), so only the checkout side is consumer work. → #147
• fetch-depth: 1 on the consumer checkouts (lecture-python.myst ci.yml:21, lecture-dp ci.yml:19, both still fetch-depth: 0); −80–95s against lecture-python.myst, whose checkout measured 101s. Land with the item above — a shallow checkout without bounded fetches makes them worse. → #148 / #149 (with #147)
• zstd layers for quantecon-build in build-containers.yml (−20–35s per PR pull); add a tag input to test-container.yml and validate a :zstd tag first. Neither docker/build-push-action call sets outputs: (.github/workflows/build-containers.yml:66-74, :108-116), so both push default gzip; test-container.yml's workflow_dispatch: takes no inputs: (:6-11) and :latest is hardcoded at :33, :99. → #150
• Concurrency cancellation: fix templates/ci.yml:24-26 to group: ci-${{ github.event.pull_request.number || github.run_id }} (it has the group and cancel-in-progress: true, not the fallback), then add the block to the consumer ci.yml files, which have none today. The fallback is load-bearing: both consumers also trigger on workflow_dispatch, so a verbatim copy puts every manual run in the empty group ci-, cancelling each other. Also closes a stale-overwrite hazard. → #147
• Paths filters (paths-ignore fail-open variant) so docs-only PRs skip the preview — templates/ci.yml has no paths key at all; plus gate both preview actions' CLI installs on pull_request: they gate on the trust check alone (preview-netlify/action.yml:56-59, preview-cloudflare/action.yml:68-71) while the deploys they feed (:63, :75) also require github.event_name == 'pull_request', so a workflow_dispatch run burns the ~25–45s install then skips the deploy. Same two steps #105 wants pinned — one PR, not two. → #147 (CLI-install gate: #105)

The --depth=1, || github.run_id and paths-ignore changes are one small actions-repo PR; the CLI-install gate folds into #105; the zstd/tag-input item stays separate because :zstd must be validated before :latest moves. Everything else is consumer-repo work, none started.

Phase 2 — 1–2 weeks: slim image, AMI bake, instant first signal

• Ship containers/quantecon-preview (~1.7 GB, zstd; drop the TeX layer; drop the jupyter metapackage — it transitively reinstalls jupyterlab; keep MKL/ipykernel for execution parity; bake pinned netlify-cli); switch lecture-dp ci/cache/publish to it; add a stack-pins-match assertion to test-container.yml. Not started — there is no containers/quantecon-preview. → #151
• Bake miniconda + env + pinned jax[cuda13]/numpyro + netlify-cli into the RunsOn AMI (the GPU repo stays on RunsOn; CPU migration evaluated and rejected) with an env_hash drift guard; flip ci.yml to setup-environment's container-mode path (verify the AMI marker file exists first; staged rollout via workflow_dispatch). → #152
• Instant placeholder PR comment (no container, no checkout — REST changed-files) with deterministic pr-N links; both actions comment only after a successful deploy (preview-netlify/action.yml:101, preview-cloudflare/action.yml:148). #14 has left this bundle (Cloudflare stable-alias fix, v0.11.1, #131); the other half — the shared comment-script extraction, PLAN item 6 — still stands and has grown: the two github-script bodies (preview-netlify/action.yml:111-186 vs preview-cloudflare/action.yml:159-242) now differ by ~12 lines (a deploymentUrl const, two title lines, two marker lines, a 7-line "This deployment" block) against item 6's recorded "only 2 lines differ". → #153
• Cache freshness warning: .cache-metadata.json at save; age check + ::warning::/step-summary in restore-jupyter-cache; cache-age-days output. Not started — the action declares exactly two outputs, cache-hit and cache-key (restore-jupyter-cache/action.yml:39-45), and no cache-metadata/cache-age string exists in the repo. No consumer repin on release: all 11 call sites float on @v0. → #154

Phase 3 — cold-case compression, incrementality, unification

• Parallel notebook pre-execution (jcache project execute --executor local-parallel) for container repos — cold 2,350s → ~950–1,550s; pilot for OOM (16 GB, no worker knob); first step of #2 → #155
• True Sphinx incrementality via mtime manipulation keyed to the cache-build SHA (touch the PR diff; full rebuild on deletions/renames/_toc.yml/_config.yml) → #156
• RunsOn Magic Cache + migrate pymyst's weekly artifact chain onto build-jupyter-cache/restore-jupyter-cache (flip cache.yml and ci.yml together; move jax/numpyro into environment.yml pip section first); ticks the PLAN.md actions/cache-on-RunsOn blocker → #157
• Cost right-sizing: g4dn.xlarge + spot for pymyst PR previews only (A/B a full cold run in 16 GB first) → #158

Decisions needed

Unanswered since July; the first gates the largest Phase 1 win.

1. HTML-only previews: should PDF/ipynb download buttons show week-old assets with a stale note, or be hidden? Is weekly detection of LaTeX breakage (with #83 alerting now working) an acceptable SLA? → #146
2. Netlify vs Cloudflare for the QuantEcon/meta#327 rollout — with CLIs pre-baked they tie on speed; the decider is Cloudflare's 25 MiB per-asset / 20k-file caps vs _pdf sizes. → Settled: Cloudflare Workers static assets (QuantEcon/infrastructure#4, #145); the asset caps move to the #145 pilot checklist.
3. Pilot gates: does 16 GB survive 4 concurrent kernels on the heaviest DP lectures; does the deployed GPU AMI contain /etc/quantecon-container? → carried in #155 and #152

Related

• PLAN.md — this plan covered its P0 (#83, done) and items 5 (done — the Cloudflare alias fix, v0.11.1), 6 (open — folded into the Phase-2 placeholder-comment line) and 9 (done).
• Prerequisites cleared: #83 (v0.11.0), #104 (v0.9.0 — the save-mode fix the lecture-dp one-liner depends on), #14 (v0.11.1).
• #18 (container-mode caching) and #2 (isolated per-lecture execution) — Phase 2 pre-baking and Phase 3 parallel execution advance both.
• #105 — overlaps Phase 1's CLI-install gate; land as one PR.
• #135 — release gating. Action changes reach consumers the moment v0 moves, since every call site floats, so the cache-age warning and the templates/ci.yml changes would ship unverified until that gate exists. Containers are looser still: quantecon-preview and the zstd images reach consumers through :latest, which no release gates — hence the :zstd-tag and stack-pins-match validations above.
• QuantEcon/meta#327 — preview unification rollout this design feeds.

Tracker structure

Converted to a native sub-issue parent per QEP-2 on 2026-08-13: every open phase item above is now a sub-issue (#146–#158, annotated inline), the infrastructure Type label is removed in that order (the QEP's exemption from "unlabelled means needs triage" keys on the sub-issue relationship), and live status moves to the sub-issues — the checklist annotations are pointers, not a second status track. The #105 CLI-install gate and the #2/#18 overlaps stay where they are; those issues are related work, not children.

This is an umbrella tracker whose live work is in sub-issues #146–#158, not one implementation. Start with PROJECT-OPTIMIZE-PREVIEWS.md and the selected sub-issue, then inspect the named entry points such as templates/ci.yml, consumer ci.yml files, and .github/workflows/build-containers.yml. Done means the chosen sub-issue's scoped change is verified and its status is tracked there.