QuantEcon / QuantEcon/quantecon-book-theme
ci: add nightly integration smoke test against lecture-python-programming
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 56
- Forks
- 24
- Avg merge
- 4h 33m
- Merged PRs (30d)
- 4
Description
Context
PR #390 migrated visual regression testing from `lecture-python-programming` to the curated `quantecon-book-theme-fixtures` repo, pinning fixtures + their build dependencies (jupyter-book, sphinx, extensions) to a specific SHA + version set. This was the right call for stability — visual diffs now attribute cleanly to theme changes.
However, this introduces a new blind spot: version drift between the pinned fixtures stack and what real QuantEcon lectures actually use today. If jupyter-book or sphinx ships a backwards-incompatible change, our CI won't notice because the fixtures repo stays on the old pinned version — but real lecture sites consuming this theme will break.
The existing `lecture-python-programming` build was useful as a real-world canary for this kind of drift. We removed it from the per-PR critical path (too slow, too noisy) but should bring it back as a non-blocking nightly smoke test.
Proposed solution
Add `.github/workflows/integration.yml` — runs nightly on a cron, builds `lecture-python-programming` from scratch with the latest released theme, fails loudly if the build breaks but does not gate any PR.
Sketch
name: Integration Smoke Test (lecture-python-programming)
on:
schedule:
- cron: \"0 6 * * *\" # daily at 06:00 UTC
workflow_dispatch:
jobs:
smoke:
runs-on: ubuntu-latest
continue-on-error: true # explicitly non-blocking
steps:
- uses: actions/checkout@v6
- name: Fetch lecture-python-programming
run: git clone --branch quantecon-book-theme https://github.com/QuantEcon/lecture-python-programming
- name: Setup Anaconda
uses: conda-incubator/setup-miniconda@v4
with:
auto-update-conda: true
python-version: \"3.13\"
environment-file: lecture-python-programming/environment.yml
activate-environment: lecture-python-programming
- name: Install theme from main
shell: bash -l {0}
run: pip install .
- name: Build HTML
shell: bash -l {0}
run: |
cd lecture-python-programming
jb build lectures --path-output ./
# Optional: open an issue if the build fails, so we don't have to
# manually scan failed workflow runs
- name: Open issue on failure
if: failure()
uses: actions/github-script@v9
with:
script: |
const date = new Date().toISOString().split('T')[0];
await github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title: \`Integration smoke failure: \${date}\`,
body: \`Nightly lecture-python-programming build broke on \${date}.\\n\\nWorkflow run: \${context.serverUrl}/\${context.repo.owner}/\${context.repo.repo}/actions/runs/\${context.runId}\\n\\nLikely cause: jupyter-book / sphinx / extension version drift. Investigate and either bump the fixtures pin in \\\`quantecon-book-theme-fixtures\\\` or fix the theme to match.\`,
labels: ['ci', 'integration-smoke']
});
Why this matters
Without this:
- A jb/sphinx version bump that breaks the theme could ship to PyPI and only get noticed when a lecture maintainer reports a broken build
- The fixtures pin would mask the issue indefinitely on PR CI
- Drift accumulates silently between releases
With this:
- Nightly heartbeat against the real, current lecture-python-programming environment
- Failures are visible (via the auto-opened issue) without spamming PR reviewers
- Fix-forward path is clear: either bump `FIXTURES_SHA` in this repo (after refreshing the fixtures repo), or patch the theme to handle the new jb/sphinx version
Tradeoffs
Pros:
- Closes the drift blind spot introduced by the fixtures migration
- Non-blocking — doesn't slow down or destabilize PR workflow
- Cheap: one nightly run ≈ 6 min of CI time
- Self-documenting failure mode via auto-opened issue
Cons:
- Bot-opened issues can be noisy if drift is frequent (mitigation: dedupe by checking for an existing open `integration-smoke` issue before creating a new one)
- Requires occasional manual triage to bump fixtures or patch the theme
- Conda + jb build is slow; can't avoid this since we're explicitly testing the full real environment
Acceptance criteria
- `.github/workflows/integration.yml` exists and runs nightly
- Workflow uses `continue-on-error: true` at the job level so failures don't show as a red ✗ on the repo's main branch status
- Failure auto-opens an issue labelled `ci` + `integration-smoke` (or comments on an existing open one rather than spamming)
- `workflow_dispatch` trigger included for manual investigation
- Documented in `docs/developer/visual-testing.md` (the "Relationship to lecture-python-programming" section) and in `quantecon-book-theme-fixtures/MIGRATION.md`
References
- #390 — fixtures-based visual regression migration that introduced this gap
- `MIGRATION.md` — the spec mentions this follow-up
- `docs/developer/visual-testing.md` — current doc references the fixtures-only flow; needs an "Integration smoke" subsection once this lands
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the existing .github/workflows/ files, docs/developer/visual-testing.md, and the referenced quantecon-book-theme-fixtures/MIGRATION.md. Add .github/workflows/integration.yml with the nightly and manual triggers, non-blocking lecture build, and failure handling described in the issue. Done means the workflow runs successfully or reports failures without gating PRs, and both documentation locations describe the integration smoke test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- anaconda, github-actions, jupyter, python
- Domain
- build-system, ci-cd, documentation, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100