QuantEcon / QuantEcon/QuantEcon.py

WASM: support_enumeration and vertex_enumeration hang in the browser — root cause is jitted generators (numba.np.linalg._LAPACK verified working)

Open
#927 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement upstream
Dominant language
Python
Stars
2.4k
Forks
2.3k
Avg merge
3d 3h
Merged PRs (30d)
3

Description

[!NOTE]
Updated 2026-08-21. Second revision today, after the cross-issue consistency pass: vertex_enumeration joins the scope — _vertex_enumeration_gen is the package's only other @jit(nopython=True) generator, so it is expected to hang for the same reason (by mechanism; not yet observed in the browser) — and the status note, the Problem section, the acceptance criteria and a new Related section are updated accordingly, now that PR #938 has merged and ci/wasm/smoke_test.py exists on main. The former "Side benefit" section is removed because its premise was wrong: _indiff_mixed_action, which makes the _LAPACK call, is already cache=True, and the # cache=True raises _pickle.PicklingError comment on the generator is a 2016 leftover unrelated to _LAPACK (cache=True on the generator works natively on Numba 0.62.1; re-enabling it is a separate maintenance item). Two permalinks and the patch count are corrected in place, and a draft of the upstream report for criterion 3 is in the comment below.

Status update (2026-08-21). The _LAPACK question is answered: on the public numba-ecosystem JupyterLite demo (Numba 0.66.0 / NumPy 2.4.6 / SciPy 1.18.0 on Emscripten) both bare np.linalg.solve inside @njit and QuantEcon's _numba_linalg_solve return correct results (https://github.com/QuantEcon/QuantEcon.py/issues/927#issuecomment-5338104706). support_enumeration still never returns, and the culprit is Numba generator functions — a minimal @njit function with yield hangs on first call. support_enumeration_gen is exactly such a generator, so the blocker has moved from LAPACK to jitted generators. vertex_enumeration / vertex_enumeration_gen is affected by the same bug: _vertex_enumeration_gen (yield at L161) is the package's only other jitted generator, and the failure happens at generator creation, before its numba.typed.Dict use is ever reached — so both game_theory enumeration routines are blocked until the upstream export is fixed. Original issue text follows; the "If it fails" fallback-solver plan is no longer needed and the acceptance criteria have been revised at the bottom.


Part of #925 (Phase 1, gated on the Phase 0 findings in #928). This is the highest-risk unknown in the browser-compliance audit.

Problem

util/numba.py imports the private API from numba.np.linalg import _LAPACK and uses it in an @overload to bind LAPACK's xgesv (Fortran, reached through SciPy's cython_lapack capsules) as a raw function address callable from nopython code. In game_theory.support_enumeration, _indiff_mixed_action calls it from inside the jitted equilibrium search, reached from the jitted generator _support_enumeration_gen.

Natively, the bound address is a machine-code pointer. Under Emscripten it must be an index into the shared WebAssembly function table, resolved across dynamically loaded side modules (SciPy's extension modules on one side, Numba's freshly linked user module on the other), with a matching signature at the call_indirect site. The QuantStack engine was explicitly designed for cross-module symbol resolution, so this may just work — but none of the (now ten) emscripten-forge numba patches mention LAPACK or linalg, and the recipe's tests never exercise np.linalg inside a jitted function.

game_theory.vertex_enumeration never touches _LAPACK, but it is driven by a jitted generator of the same kind — _vertex_enumeration_gen, returned by vertex_enumeration_gen and consumed by vertex_enumeration — which is why it is in scope now that the hang is traced to generators rather than to the LAPACK binding.

How to test

The same _LAPACK mechanism backs Numba's own np.linalg.solve support, so the clean experiment (part of the Phase 0 smoke suite, #928) is: run np.linalg.solve inside a scratch @njit function in the browser. If that works, QuantEcon's helper almost certainly works too; if it fails, both fail together.

If it fails

In order of preference:

  1. A pure-Numba LU solve with partial pivoting behind a sys.platform == "emscripten" gate. support_enumeration solves small dense (k+1)×(k+1) systems for support size k, so a hand-written nopython solver is entirely adequate there.
  2. Report the gap upstream to the emscripten-forge / Numba WASM effort — Numba's own linalg support has the same dependency, and the authors are actively expanding coverage.

Related

  • PR #938 (merged 2026-08-21): ci/wasm/environment.yml, ci/wasm/smoke_test.py, ci/wasm/test_jupyterlite.py and the native ci_wasm_smoke.yml job are now on main. test_support_enumeration and test_vertex_enumeration in smoke_test.py are unguarded and will hang in the browser, and test_jupyterlite.py runs test_support_enumeration in its shared kernel; guarding them with something that does not execute the test, and adding the minimal generator test from criterion 2, is part of the #933 runner work.
  • #933: browser runner job; carries the hang-guard and isolation-test notes for this issue.
  • #944: the sibling upstream blocker (cache=True callers of cache-restored functions fail with "no compiled object yet", reported upstream as emscripten-forge/recipes#6309). Different mechanism, same "wait for upstream" posture.
  • #925 (umbrella) and #928 (Phase 0 results table, where the support_enumeration and bare-generator rows belong).

Acceptance criteria

  • In-browser verdict recorded: np.linalg.solve inside @njit works on Emscripten (LAPACK binding resolves correctly)
  • Minimal jitted-generator repro added to the smoke suite (ci/wasm/smoke_test.py, on main since PR #938) so the mechanism is isolated the way the bare np.linalg.solve test isolates LAPACK
  • Generator hang reported upstream to the emscripten-forge / Numba WASM effort with the minimal repro (draft report in the comment below)
  • Decide whether support_enumeration and vertex_enumeration need an Emscripten-gated non-generator path, or whether we wait on the upstream fix
  • In-browser tests cover support_enumeration and vertex_enumeration (test_support_enumeration and test_vertex_enumeration in ci/wasm/smoke_test.py, currently unguarded) so regressions are caught once they run

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with ci/wasm/smoke_test.py and the existing test_support_enumeration and test_vertex_enumeration cases. Add the minimal jitted-generator reproduction, report it upstream with the browser result, and determine whether the enumeration paths need an Emscripten-specific treatment or should wait for the upstream fix. Done means the smoke tests isolate the hang and the upstream report and project decision are recorded.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, wasm
Domain
backend, ci-cd, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.