QuantEcon / QuantEcon/actions

test-containers-lectures: absorb transient raw-GitHub 429s when building lecture repos

Open
#102 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

good first issue infrastructure
Dominant language
Shell
Stars
0
Forks
1
Avg merge
32m
Merged PRs (30d)
3

Description

A test-containers-lectures.yml run failed on lectures/hansen_singleton_1983.md in QuantEcon/lecture-python.mystCellExecutionError / HTTP Error 429: Too Many Requests: a hidden cell fetches its dataset from raw GitHub at build time (lectures/hansen_singleton_1983.md:1432-1439).

The matrix builds several repos at once from GitHub's shared runner IPs, so raw GitHub rate-limits it — non-deterministically: the same notebook passed on the other container in the same run. It surfaced during the Anaconda 2026.06 migration (#95), but the cause is the fetch, not the migration or pandas.

Lecture-side fixes do not work: local _static/ paths break Colab, since QuantEcon/lecture-python.notebooks is a flat .ipynb directory with no _static/ (QuantEcon/lecture-python.myst#972, closed for that reason). It is not lecture-specific either — lectures/hansen_singleton_1982.md:1000, lectures/pandas_panel.md:68,188,505 and lectures/phillips_drifts_volatilities.md:103 fetch over raw GitHub too. Moving datasets to data-lectures does not help: pandas_panel.md already reads from that repo over the same github.com/.../raw/ host.

No retry, backoff or 429 handling exists here. The only mitigation, per-repo serialization at .github/workflows/test-containers-lectures.yml:46-48 (v0.7.0, re-scoped by run id in v0.9.0, #103), pairs the two containers for one repo; the three repos at :55-60 still build in parallel. Flakes also file tracker noise: on any non-workflow_dispatch failure (:138), notify-failure (:132-154) runs scripts/create-ci-failure-issue.sh → "🔴 CI failure: Test Container Builds".

Fix: retry the HTML build stage. With execute_notebooks: "cache" (lectures/_config.yml:21) and lecture-repo/_build/.jupyter_cache persisting across steps in the job — why stages 2 and 3 already skip execution — a retry re-executes only the failed notebooks, not the full 30-60 minute run, so no need to gate on the 429 string.

  • Give the "Build HTML (Execute Notebooks)" step (:95-102) an id and continue-on-error: true, then a conditional second attempt on steps.<id>.outcome == 'failure'
  • Give the retry a distinct failure-artifact-namebuild-lectures uploads execution reports under that name on failure (build-lectures/action.yml:38,179), so a repeated name collides with the first attempt's artifact
  • Validate with a maintainer workflow_dispatch after merge — it triggers only on workflow_run and workflow_dispatch (:22-27), so a PR cannot exercise it

Acceptance: a single transient 429 no longer fails the matrix; a persistent failure still does.

While in this workflow: restore the missing matrix repo

lecture-python-programming was dropped from the matrix (:54-64) in an unannounced swap in 81e96ff — unlike lecture-jax, which is commented out at :61-64 with a TODO and an issue link. containers/VALIDATION.md:25,42,46,50 and TESTING.md:11 still advertise the 4-repo matrix, so the only place lecture content executes on the images silently lost a quarter of its coverage while the docs kept claiming it.

It belongs in this PR rather than its own: it is four lines in the file you are already editing, and the repo has six build-time raw-GitHub fetches plus yfinance calls, so the retry above is the prerequisite for adding it without immediately reintroducing the flake. Note it is also the repo pinning anaconda=2026.07 against the containers' 2026.06.

  • Add the lecture-python-programming matrix row (current repo name, no .myst suffix)
  • Correct the 4-repo claims in containers/VALIDATION.md and TESTING.md to match

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with .github/workflows/test-containers-lectures.yml, especially the Build HTML (Execute Notebooks) step and matrix rows, then inspect build-lectures/action.yml for artifact naming. Update the workflow to retry failed HTML builds and restore the lecture-python-programming row, then align the four-repo statements in containers/VALIDATION.md and TESTING.md. Done means transient 429s are absorbed while persistent failures still fail and the matrix documentation matches.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions
Domain
ci-cd, documentation
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.