QuantEcon / QuantEcon/data-lectures
Work plan — P4 dynamic snapshots: lecture-wasm adoption of the business_cycle set
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1
- Forks
- 1
- Avg merge
- 58m
- Merged PRs (30d)
- 18
Description
Session bootstrap: opened at the end of the 2026-09-01 session, which ran with no plan (its work is the session's trace — PRs #106–#117 here, plus issues in three other repos), as the handover to the next session. No predecessor. This body is the single source of truth for the session, revised in place against live state before working it; comments below are the revision log.
The programme this plan sits in: the static migration completed 2026-08-18 (40 of 40 datasets), and the P4 dynamic-snapshot pilot is the active front — reframed on 2026-09-01 from "a UNRATE twin" to "the business_cycle set", because the lecture that needs a twin is excluded from lecture-wasm for want of one and a partial twin buys it nothing. Cross-repo tracker: QuantEcon/workspace-lectures#14. Roadmap: PLAN.md Phase 5 and Phase 8 P4.
Where we stand (verified 2026-09-01 06:45 UTC)
| Fact | Measured | How |
|---|---|---|
main |
a0e1415 (#117 merged) |
git log |
| Open PRs here | none | gh pr list |
| Dynamic snapshots | 4 — business_cycle_data.csv, unemployment_rate_annual.csv, private_credit_to_gdp.csv (annual), us_business_cycle_monthly.csv (monthly) — all retrieved 2026-09-01, verified, none due |
scripts/snapshots.py list / due |
refresh-snapshots |
last dispatch green: two canary legs (one per builder), FRED leg fetched and validated, nothing due | run 33478359372 |
audit-dashboard, consumed-file-check on main |
green | latest runs 06:36 UTC |
| Live dashboard | 40 static files, 40 migrated, 23 live-API lectures, 1 orphan (was 24 on 2026-08-31 — Track X landed in the lecture repos today), 2 URL forms | audit.json generated 2026-09-01 |
| Consumers of the four snapshots | none; lecture-python-intro keeps its live calls by decision |
manifests |
QUANTECON_SERVICES_PAT |
reaches this repo (added 2026-09-01); refresh PRs trigger the required check | #112's checks ran |
| Cross-repo | QuantEcon/lecture-wasm#70 open (stage 1 recorded there); QuantEcon/lecture-python-advanced.myst#381 open (the last vestigial pandas-datareader install) |
gh |
| Naming policy | #113 open for discussion; the three new filenames are provisional and free to change while consumers is empty |
#113 |
| Open issues here | 14: #122 #121 #120 #119 #113 #88 #87 #66 #40 #39 #37 #35 #26 #14 — of which #35 #37 #39 #87 are standing records by design, and #119–#122 are the Phase 5 children of the #14 tracker (filed 2026-09-02) | gh issue list |
Work blocks, in dependency order
A. Naming decision (#113). Gate: Matt's call on the strawman (name the variable; frequency when ambiguous; a composite named for its topic; no source prefix; rename only while consumers: []). Affects the three provisional names. Not strictly blocking B — see the front of the plan — but it is the last free moment for the names, since C is the point of no return.
B. Stage 2 — the lecture-wasm PR (lecture-wasm#70). In lectures/business_cycle.md: every wb.data.DataFrame(...) and web.DataReader(...) becomes a pd.read_csv('https://raw.githubusercontent.com/QuantEcon/data-lectures/main/lectures/<file>') with the selection applied in pandas (.loc[[...]], a date slice; the composite's M0892AUSM156SNBR column keeps the lecture's existing rename to UNRATE); the %pip install wbgapi … pandas-datareader cell and both imports go; the API-teaching cells (wb.series.info, wb.series.metadata.get) become non-executing code blocks with a one-line note that the intro edition runs them live; lectures/myst.yml re-enables business_cycle; README drops it from the unsupported list and retires the dead "sync from intro's wasm branch" rule (that branch is 125 commits behind and the last ten PRs edited lectures/ directly). Gate: the Netlify preview runs the lecture in-browser end to end (JupyterLite "Run all" stops at the first failing cell, so a partial repoint proves nothing). Work from the ../lecture-wasm clone (level with origin/main at aecec4d on 2026-09-01; re-fetch first).
C. Stage 3 — the flip, here. On all four manifests: consumers: [{repo: QuantEcon/lecture-wasm, file: lectures/business_cycle.md, on_refresh: rebuild}]; migration.yml records → repointed; CATALOG.md regenerated. Do the both-directions strict-audit dry run before pushing (PLAN P3 precedent: landed → exit 1, repointed → exit 0). lecture-wasm publishes on push to main, so no repoint/publish/delete split, and nothing is deleted. Gate: B merged. This is where names freeze.
D. Consumer fan-out (PLAN Phase 5, the one unticked automation box). C creates the first snapshot with a consumer, so the on_refresh: rebuild dispatch in refresh-snapshots.yml becomes real: on a merged refresh, dispatch lecture-wasm's build (needs the PAT, which is present). Gate: C. Small.
E. Housekeeping, any time. Merge QuantEcon/lecture-python-advanced.myst#381. Close #26 as decided (Matt's call; the decision is recorded there). #14's remaining half is now its four sub-issues (filed 2026-09-02, #14 conformed to the status-projects tracker contract the same day): #119 manifest-driven validate() reading the sidecar's schema as its spec plus the PR-validation workflow, and the three schema decisions #120 column pattern, #121 known_nulls exact-vs-ceiling, #122 dtype vocabulary. That is the next Phase 5 slice after D; landing any of it re-dates #14's stamp and closes the child. #88 and #40 stay low priority.
Front of the plan
B leads, with A run in parallel rather than ahead: start the wasm draft now and accept that C may rename a file or two — a rename before C costs a manifest edit, a git mv, a regenerated catalog and nothing else, because nothing reads the files yet. Waiting for #113 to conclude would idle the pilot on a discussion with no deadline. If Matt prefers the names settled first, A leads and B waits; either order is fine, and the plan should record which was chosen.
Explicitly not doing
- Adopting the snapshots in
lecture-python-intro. Decided 2026-09-01: intro keeps every live call; the API is the lesson there. A snapshot twin sits beside it (in wasm) rather than replacing it. - The other live-API lectures (23 today, including
unemployment_linear/unemployment_shocksin python.myst, which read UNRATE over the same window the composite carries). Candidates for later adoption; not this plan. - A generic builder harness (
scripts/_builder.py, #14)._fred.pyis a fetch library; the generic validate/write harness still waits for a second source class, as #14 says. pandas_datareader.macroas a fetch layer — no, until a builder genuinely needs FRED + Eurostat + OECD in one frame (#26).- qeld (#66,
PLAN-QELD-PACKAGE.md): decisions refreshed 2026-09-01; no implementation started; not this plan. - Renaming
business_cycle_data.csv— keep, by decision, until #113 settles a policy.
Learned this session — things a fresh session would otherwise rediscover
- FRED stalls custom user agents from GitHub-hosted runners (
qeld-builder, evenMozilla/5.0) and answers urllib's default and curl in ~50 ms; the same requests all succeed from a workstation. Recorded inbuilders/_fred.py's docstring; found by a probe run on a runner (#116, closed), because it reproduces nowhere else. workflow_dispatchonly runs workflows that exist on the default branch. A branch-only diagnostic has to ride apull_requesttrigger (a draft PR) instead.- The stamp's read-back guard earns its keep: it refused a date-shaped
date_range.endon the first multi-file stamp (YAML reads it back as a date), and refused a foreign summary on the firstpr-bodytest. - Manifest prose must not embed vintage facts (an end year, an observed range, a row count): the stamp rewrites fields, not sentences — the first refresh PR shipped a stale column description. Rule now in AGENTS.md and
builders/_template.py. - The wasm README's "never edit lectures here, sync from the
wasmbranch" rule is dead — see block B. - The
datasetdispatch input and per-builder canary legs were both wrong on the first outing and are fixed (#117); a set-writing builder now runs once.
Resume pointer
Next session: resume this plan (re-measure the table — the Monday 06:17 UTC canary will have run), then block B as a draft in ../lecture-wasm for approval, following the drafts-then-"go" pattern this repo's sessions use.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Re-measure the plan and re-fetch the ../lecture-wasm clone before starting. Read lectures/business_cycle.md, lectures/myst.yml, and the wasm README, then inspect the four snapshot manifests, migration.yml, CATALOG.md, and refresh-snapshots.yml. Done means the wasm lecture runs end to end in its Netlify preview, the PR merges, manifests and catalog are updated, and the refresh dispatch is exercised.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook, pandas, python, wasm
- Domain
- ci-cd, data, documentation
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100