QuantEcon / QuantEcon/actions

Stale-asset republication: default templates to cache-type execution, or prune html inside restore-jupyter-cache

Open
#160 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
0
Forks
1
Avg merge
32m
Merged PRs (30d)
3

Description

The bug class the templates would carry into the migration

Restoring a full _build and building over it republishes files that have been deleted from source. Sphinx copies html_static_path assets into _build/html/_static but never removes ones that no longer exist in source, so a dataset or asset deleted on Tuesday keeps being served by every preview and publish that restores Monday's cache — until a clean cache rebuild happens to intervene. This is not hypothetical: it made deletion verification unreliable across the Track C dataset migration, and QuantEcon/workspace-lectures#41 is currently rolling a consumer-side jb clean . --html step across 9 lecture repos as the fix for the dawidd6-artifact world (validated live in QuantEcon/lecture-python-advanced.myst#374 — a restored cache that physically contained three deleted files produced a preview that 404'd all three while serving controls at 200).

The migration surface here inherits the same bug: templates/ci.yml pins cache-type: 'build' (full _build, html included), and neither restore-jupyter-cache, build-lectures, nor the docs prune or mention stale assets. A repo cutting over by template today lands back in the pre-fix state, and the per-repo workflow prune steps from the rollout would be removed at exactly that moment.

The structural fix is already half-built

restore-jupyter-cache supports cache-type: 'execution' (.jupyter_cache only). Run that way, every consumer rebuilds _build/html from its own checkout: the stale-asset class cannot exist, no prune is needed anywhere, and the guarantee is immune to which vintage of cache gets matched (relevant because prefix-matching means an old entry can be restored silently). The restored html contributes nothing to build speed anyway — a fresh checkout's mtimes force Sphinx to rewrite every page — so the real cost of execution-only mode is redoing the pdflatex/jupyter render passes, not notebook execution.

Proposal

  1. Flip the site-publishing templates (ci.yml, publish.yml) to cache-type: 'execution' as the default, with a short note in the template explaining why (deletions must take effect at the next build, not the next weekly cache).
  2. For any repo that measurably needs build mode (render passes too slow), add a prune-html input to restore-jupyter-cache — default true — that runs the equivalent of jb clean . --html after restore, so the safeguard lives in one composite action rather than N workflow files. (jb clean needs no _config.yml at the restore path; verified with jupyter-book 1.0.4.post1 that it removes exactly <path>/html — in _build terms: html only, .jupyter_cache/latex/jupyter untouched.)
  3. Note in the migration guide that repos carrying the interim workflow-level prune step from the rollout should drop it at cutover — the action supersedes it.

Filed from the discussion on QuantEcon/lecture-python-programming#616 (repo 2 of the rollout), where the interim prune is landing now; the rollout continues as the bridge until cutover.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading templates/ci.yml, publish.yml, and the restore-jupyter-cache action to compare the current build-cache behavior with the execution mode. Check the migration guide and existing workflow-level prune steps. Done means the selected default safeguards deleted assets after cache restore and the migration guidance explains how interim prune steps are replaced.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, jupyter, shell
Domain
build-system, ci-cd, documentation
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.