QuantEcon / QuantEcon/QuantEcon.py
WASM: audit 32-bit intp behaviour on wasm32 (overflow guards, simplex_index wrapping, dtypes)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.4k
- Forks
- 2.3k
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 3
Description
[!NOTE]
Updated 2026-08-21 (second edit). Tracker reconciliation: PR #942 is still open with a live "Fixes #929" and is to be closed unmerged; PR #943 is stacked on it and carries the sameint64widening, so it must be rebased ontomainkeepingintpbefore merge. The earlier note and the #944 summary below now attribute the browser failure to the cache-hit/cache-miss linking bug in the emscripten-forge Numba build (emscripten-forge/recipes#6309), not tocomb_jit's eager signature, and the grep-sweep task namesk_array_rank_jitas sharing theintpboundary. Labels changed fromteststodocumentation.
[!NOTE]
Updated 2026-08-21. The original body described three problems; review of #942 showed two of them were wrong and the third is a documentation task, not a code change. Thecomb_jit/simplex_griduse ofnp.intpis intentional and should be kept — see below. The genuine WASM blocker surfaced by #942 (cache=Truecallers failing to link cache-restored callees on the emscripten-forge Numba build) is tracked separately in #944. The original text is preserved in the edit history.
Part of #925 (Phase 1). Verification runs on the Phase 0 deployment (#928).
Background
Emscripten/wasm32 is a 32-bit platform: np.intp and NumPy's default integer are int32, so the library behaves as it would on 32-bit Linux. This issue audits what that means for QuantEcon.py.
Findings
comb_jit / simplex_grid — intp is correct by design; document the boundary, don't widen it. comb_jit is typed intp(intp, intp) and returns 0 when the result would exceed np.iinfo(np.intp).max — 2³¹−1 in the browser instead of 2⁶³−1. Its result is consumed as an array size (np.empty((L, m)) in simplex_grid) and as a rank in k_array_rank_jit, so intp is exactly the right width: it is the largest value that can be a shape on the platform. Widening to int64 would let L ∈ [2³¹, 2⁶³) past the guard and into np.empty, trading the current clean ValueError('Maximum allowed size exceeded') for a wrap or an obscure allocation failure. Nothing real is lost either: with m ≥ 2, any grid with L ≥ 2³¹ needs ≥ 16 GB, four times wasm32's whole address space. The only action is to state the platform-dependent boundary in the docstrings.
simplex_index does not wrap. The original body claimed it accumulates It is a pure-Python function that uses comb_jit terms with wrapping intp arithmetic.num_compositions (SciPy's exact comb), so it works in unbounded Python ints. No change needed.
Test assertions are already platform-agnostic. The original body asked for dtype == np.int64 assertions to be made platform-agnostic.TestCombJit already keys off np.iinfo(np.intp), and the only int64 assertions in the suite (test_game_converters.py) round-trip an explicit dtype=np.int64 argument. No change needed.
Result dtypes shift (informational). Jitted code allocating with np.int_ / np.intp (the draw overload in random/utilities.py, index arrays in _gridtools.py) returns int32 arrays in the browser. That is correct behaviour. Where the contract is int64 via explicit i8 gufunc signatures (sample_without_replacement), NumPy's safe casting absorbs int32 inputs. Worth one sentence in the WASM docs; no code change.
Tasks
- Docstrings for
comb_jit,num_compositions_jitandsimplex_grid: say the overflow boundary isnp.iinfo(np.intp).maxand therefore platform-dependent (2³¹−1 on wasm32) - Document the wasm32 limits in the JupyterLite user-facing notes: moved grid-size boundary, int32 default integer, 2–4 GB memory cap shared with the whole kernel (coordinate with #931)
- Grep sweep of remaining
intparithmetic sites — starting withk_array_rank_jit(quantecon/util/combinatorics.py:103), which callscomb_jitand so shares the samenp.iinfo(np.intp).maxboundary — to confirm none has a non-size/index use whereint64would be the better contract (expected outcome: none)
Related
- #942 — proposed widening
comb_jittoint64; superseded by the analysis above and to be closed unmerged. Its body still opens with "Fixes #929", so merging it would both reverse the keep-intpdecision and auto-close this issue. - #943 (lazy Numba compilation for #930) — stacked on #942: its commit list includes #942's
97518e0, and its own commit replaces the eager signature with lazy@jit(nopython=True, cache=True)plusnp.int64()casts, so merging it as-is would land theint64widening (tests, docstrings and thetest_simplex_grid_comb_int64smoke test attributed to this issue). It must be rebased ontomainwith97518e0dropped andintpretained (np.iinfo(np.intp).max, no widening casts) before merge. - #944 —
simplex_gridfails in the browser withRuntimeError: no compiled object yet for comb_jit. The cause is a cache-hit/cache-miss linking bug in the emscripten-forge Numba build, reported upstream as emscripten-forge/recipes#6309: patch 0006's_serialize_wasm_linking_librariescalls_get_compiled_object()on acache=Truecallee restored from the persistent cache, whose object_object_getbuffer_hookhas already consumed. It triggers when a cache-misscache=Truecaller links a cache-hitcache=Truecallee in the same warm session, so it needs a warm cache and does not reproduce in a cold single session;comb_jit's eager signature is not the cause (it only makescomb_jitthe deterministic trigger). This, not integer width, is what currently blockssimplex_gridin the browser, and the same cached→cached pattern exists ink_array_rank_jit → comb_jitand_lemke_howson_tbl → _pivoting/_lex_min_ratio_test.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the docstrings for comb_jit and num_compositions_jit in quantecon/util/numba.py, simplex_grid in quantecon/_gridtools.py, and k_array_rank_jit in quantecon/util/combinatorics.py. Review the JupyterLite user-facing notes and grep remaining intp arithmetic sites. Done means the platform-dependent limits and result-dtype behavior are documented and the audit finds no non-size/index use requiring a different contract.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python, wasm
- Domain
- documentation, web-dev
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100