test-jax-install.py does not assert the GPU — a silent CPU fallback passes CI in all five repos

Open Beginner friendly
#384 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
84/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
github-actions, python

Research direction

Start with scripts/test-jax-install.py in each of the five repositories listed in the issue, comparing the existing jax.devices() call and PRNG-key variant. Run the installation check in the GPU workflow and verify that it fails when JAX uses the CPU and passes when the backend is GPU-backed.

Written by the indexing model from the issue text.

Description

bug testing

The GPU check that every JAX workflow runs prints the device list and never asserts it. A build whose driver, CUDA install or GPU passthrough had quietly fallen back to the CPU would still pass, and would publish CPU-timed output — which is the one failure GPU CI exists to prevent, since these are GPU-acceleration lectures that time themselves and print their speed gains in the published text.

Verified against main in all five repositories on 2026-09-04.

Where it is

Repository Variant
lecture-jax jax.random.PRNGKey(0)
lecture-python.myst jax.random.key(0)
lecture-python-programming jax.random.PRNGKey(0)
lecture-stats jax.random.PRNGKey(0)
iuj_feb_2026 jax.random.PRNGKey(0)

All five copies of scripts/test-jax-install.py are functionally identical — the only differences are the legacy versus current PRNG key call and a trailing newline. None of them contains an assert, a raise or a sys.exit, so the script cannot fail for a wrong-platform reason. Its own comment says the jitted matmul "will likely run on GPU (if available)", which is exactly the uncertainty the check should be removing.

nvidia-smi runs as a separate workflow step, so it proves the host can see a GPU. It does not prove JAX bound to one. Those are different failures: a driver too old for the installed jax[cuda13] wheel gives a perfectly healthy nvidia-smi alongside a CPU-only JAX.

The fix

One line, after the existing jax.devices() call:

assert jax.default_backend() == "gpu", f"JAX fell back to {jax.default_backend()}: {devices}"

jax.default_backend() returns 'gpu' for CUDA and ROCm alike, so it needs no vendor branch. A stricter variant that also names the device is assert any(d.platform == "gpu" for d in jax.devices()).

Why it is worth doing now

Two reasons beyond tidiness.

The first is that the failure is silent by construction. Nothing in the current pipeline distinguishes "the GPU worked" from "JAX ran on the CPU and the lectures published slower numbers", and the timings are part of the rendered output rather than a side effect of it.

The second is that QuantEcon/project-compute (private) is planning a self-hosted GPU runner pilot on hardware we own, where a CPU fallback becomes materially more likely than it is on the rented RunsOn images — the driver becomes ours to maintain, and jax[cuda13] needs an NVIDIA driver of at least 580 and SM 7.5 or newer. The check that ought to catch that regression currently cannot. This surfaced while surveying per-job isolation for that pilot.

Filed here rather than as five issues because the change is the same in all five repositories.

Dominant language
No language data
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from QuantEcon/meta

All issues in QuantEcon/meta

Similar issues

More DevOps issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.