Stale-asset republication: default templates to cache-type execution, or prune html inside restore-jupyter-cache
Nobody has claimed this yet.
- Dominant language
- Shell
- Stars
- 0
- Forks
- 1
- Avg merge
- 32m
- Merged PRs (30d)
- 3
Description
The bug class the templates would carry into the migration
Restoring a full _build and building over it republishes files that have been deleted from source. Sphinx copies html_static_path assets into _build/html/_static but never removes ones that no longer exist in source, so a dataset or asset deleted on Tuesday keeps being served by every preview and publish that restores Monday's cache — until a clean cache rebuild happens to intervene. This is not hypothetical: it made deletion verification unreliable across the Track C dataset migration, and QuantEcon/workspace-lectures#41 is currently rolling a consumer-side jb clean . --html step across 9 lecture repos as the fix for the dawidd6-artifact world (validated live in QuantEcon/lecture-python-advanced.myst#374 — a restored cache that physically contained three deleted files produced a preview that 404'd all three while serving controls at 200).
The migration surface here inherits the same bug: templates/ci.yml pins cache-type: 'build' (full _build, html included), and neither restore-jupyter-cache, build-lectures, nor the docs prune or mention stale assets. A repo cutting over by template today lands back in the pre-fix state, and the per-repo workflow prune steps from the rollout would be removed at exactly that moment.
The structural fix is already half-built
restore-jupyter-cache supports cache-type: 'execution' (.jupyter_cache only). Run that way, every consumer rebuilds _build/html from its own checkout: the stale-asset class cannot exist, no prune is needed anywhere, and the guarantee is immune to which vintage of cache gets matched (relevant because prefix-matching means an old entry can be restored silently). The restored html contributes nothing to build speed anyway — a fresh checkout's mtimes force Sphinx to rewrite every page — so the real cost of execution-only mode is redoing the pdflatex/jupyter render passes, not notebook execution.
Proposal
- Flip the site-publishing templates (
ci.yml,publish.yml) tocache-type: 'execution'as the default, with a short note in the template explaining why (deletions must take effect at the next build, not the next weekly cache). - For any repo that measurably needs
buildmode (render passes too slow), add aprune-htmlinput torestore-jupyter-cache— defaulttrue— that runs the equivalent ofjb clean . --htmlafter restore, so the safeguard lives in one composite action rather than N workflow files. (jb cleanneeds no_config.ymlat the restore path; verified with jupyter-book 1.0.4.post1 that it removes exactly<path>/html— in_buildterms: html only,.jupyter_cache/latex/jupyteruntouched.) - Note in the migration guide that repos carrying the interim workflow-level prune step from the rollout should drop it at cutover — the action supersedes it.
Filed from the discussion on QuantEcon/lecture-python-programming#616 (repo 2 of the rollout), where the interim prune is landing now; the rollout continues as the bridge until cutover.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading templates/ci.yml, publish.yml, and the restore-jupyter-cache action to compare the current build-cache behavior with the execution mode. Check the migration guide and existing workflow-level prune steps. Done means the selected default safeguards deleted assets after cache restore and the migration guidance explains how interim prune steps are replaced.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, jupyter, shell
- Domain
- build-system, ci-cd, documentation
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100