Exercise design: should lectures ship function skeletons and unit tests?

Start by reading dependency issue #340 and inspecting the sphinx-tojupyter execution step in lecture-python-programming, the proposed pilot repository. Compare its exercise structure with the described skeleton-and-test model; done means the project has decided whether and where to pilot executable checks, including the intended reader and build benefits.

Note: originally filed as a book recommendation for Mathematical Logic through Python. Reviewed and rewritten in place, July 2026 — the subject is not ours, but the book's exercise design is worth considering, and that is what this issue now tracks.

Where we are now

Our exercises are a prose prompt followed by a dropdown solution — {exercise} with a :label:, then {solution-start} with :class: dropdown. The reader attempts it, reveals our answer, and compares by eye. There is no way to check an attempt programmatically.

Across lecture-python-intro and lecture-python-programming there are zero occurrences of assert, pytest or unittest. The exercise apparatus is entirely prose plus a printed answer.

That is fine for exercises whose output is a figure or an interpretation. It is weaker for exercises whose output is a function, where "did I get it right?" has a definite answer the reader currently cannot obtain.

The model

Mathematical Logic through Python, Yannai A. Gonczarowski and Noam Nisan, Cambridge University Press, 2022. All 13 chapters are free as pre-publication PDFs (personal use, non-commercial, no redistribution); the published edition is paid.

Each chapter supplies the theory for an associated programming project. The reader downloads a package of documented function skeletons plus a comprehensive unit-test suite, implements the skeletons, and runs the tests. The projects accumulate chapter over chapter until the reader has built a working proof system from nothing. It is the same method as nand2tetris, which Nisan co-created.

Nisan is not a stranger to this codebase — he is already the reference behind quantecon/game_theory/support_enumeration.py, lemke_howson.py and vertex_enumeration.py in QuantEcon.py, and their GameTheory.jl counterparts.

What might transfer

1. Skeleton plus test, rather than prompt plus answer. Give the reader a signature and a docstring, then a test cell. In MyST this needs no new machinery:

```{exercise}
:label: ex_gini

Implement the function below so that it returns the Gini coefficient of `w`.

```{code-cell} ipython3
def compute_gini(w):
    """
    Compute the Gini coefficient of a sample of wealth levels.

    Parameters
    ----------
    w : array_like
        A one-dimensional sample of non-negative wealth levels.

    Returns
    -------
    float
        The Gini coefficient, in [0, 1].
    """
    pass  # your code here
```

and after the solution, a check:

```{code-cell} ipython3
:tags: [hide-input]

assert abs(compute_gini(np.ones(100))) < 1e-10       # perfect equality
assert abs(compute_gini(np.array([0.0] * 99 + [1.0])) - 0.99) < 1e-10
```

Be clear about who this helps and how. In the published HTML the test validates our solution, not the reader's — the reader's attempt is never executed. Its value there is that a broken solution fails the build. For a reader who downloads the notebook — which every lecture offers — the same cell becomes a genuine pass/fail on their own attempt. Both are worth having, but they are different benefits, and the second is the one the book is really demonstrating.

2. Cumulative projects. Their chapters build one artifact end to end; our exercises are self-contained per lecture. Worth weighing against the course-shaped publications direction in #312 — a course could carry a spine project that each lecture advances, instead of a set of unconnected drills.

3. Skeletons as a difficulty dial. A signature plus a docstring is a substantial hint. It lets an exercise be hard without being unapproachable, which is hard to tune with a prose prompt alone.

Dependency: this needs #340 first

A test cell only gates anything if a failing cell actually fails the build. Per #340, it does not in three repos — lecture-python-intro, lecture-python.zh-cn and lecture-intro.zh-cn are missing -n -W on the sphinx-tojupyter execution step, so a CellExecutionError is a non-fatal warning there. Adding asserts to an intro lecture today would produce tests that can fail silently, which is worse than no tests. #340 should land before any pilot in those repos.

Caveats

• The free chapters are personal-use and non-redistributable, and the code package's licence has not been checked. Anything adopted should be the method, not the material.
• Test cells add execution time and a maintenance surface: a stale test breaks the build. Arguably a feature, but it should be chosen deliberately.
• Not every exercise suits this. Ones asking for a plot, an interpretation or a derivation have no testable return value.

Questions

• Worth piloting on one lecture? lecture-python-programming is the natural candidate — its exercises are most often "write this function", and it already has -W on the execution step, so the gate works there today.
• Should the test cell be hidden (hide-input, so it does not give away the shape of the answer) or visible (so the reader sees what correctness means)?
• Does the cumulative-project idea belong here, or as a separate question against #312?