QuantEcon / QuantEcon/QuantEcon.py

WASM: audit 32-bit intp behaviour on wasm32 (overflow guards, simplex_index wrapping, dtypes)

Open
#929 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

documentation maintenance
Dominant language
Python
Stars
2.4k
Forks
2.3k
Avg merge
3d 3h
Merged PRs (30d)
3

Description

[!NOTE]
Updated 2026-08-21 (second edit). Tracker reconciliation: PR #942 is still open with a live "Fixes #929" and is to be closed unmerged; PR #943 is stacked on it and carries the same int64 widening, so it must be rebased onto main keeping intp before merge. The earlier note and the #944 summary below now attribute the browser failure to the cache-hit/cache-miss linking bug in the emscripten-forge Numba build (emscripten-forge/recipes#6309), not to comb_jit's eager signature, and the grep-sweep task names k_array_rank_jit as sharing the intp boundary. Labels changed from tests to documentation.

[!NOTE]
Updated 2026-08-21. The original body described three problems; review of #942 showed two of them were wrong and the third is a documentation task, not a code change. The comb_jit / simplex_grid use of np.intp is intentional and should be kept — see below. The genuine WASM blocker surfaced by #942 (cache=True callers failing to link cache-restored callees on the emscripten-forge Numba build) is tracked separately in #944. The original text is preserved in the edit history.

Part of #925 (Phase 1). Verification runs on the Phase 0 deployment (#928).

Background

Emscripten/wasm32 is a 32-bit platform: np.intp and NumPy's default integer are int32, so the library behaves as it would on 32-bit Linux. This issue audits what that means for QuantEcon.py.

Findings

comb_jit / simplex_gridintp is correct by design; document the boundary, don't widen it. comb_jit is typed intp(intp, intp) and returns 0 when the result would exceed np.iinfo(np.intp).max — 2³¹−1 in the browser instead of 2⁶³−1. Its result is consumed as an array size (np.empty((L, m)) in simplex_grid) and as a rank in k_array_rank_jit, so intp is exactly the right width: it is the largest value that can be a shape on the platform. Widening to int64 would let L ∈ [2³¹, 2⁶³) past the guard and into np.empty, trading the current clean ValueError('Maximum allowed size exceeded') for a wrap or an obscure allocation failure. Nothing real is lost either: with m ≥ 2, any grid with L ≥ 2³¹ needs ≥ 16 GB, four times wasm32's whole address space. The only action is to state the platform-dependent boundary in the docstrings.

simplex_index does not wrap. The original body claimed it accumulates comb_jit terms with wrapping intp arithmetic. It is a pure-Python function that uses num_compositions (SciPy's exact comb), so it works in unbounded Python ints. No change needed.

Test assertions are already platform-agnostic. The original body asked for dtype == np.int64 assertions to be made platform-agnostic. TestCombJit already keys off np.iinfo(np.intp), and the only int64 assertions in the suite (test_game_converters.py) round-trip an explicit dtype=np.int64 argument. No change needed.

Result dtypes shift (informational). Jitted code allocating with np.int_ / np.intp (the draw overload in random/utilities.py, index arrays in _gridtools.py) returns int32 arrays in the browser. That is correct behaviour. Where the contract is int64 via explicit i8 gufunc signatures (sample_without_replacement), NumPy's safe casting absorbs int32 inputs. Worth one sentence in the WASM docs; no code change.

Tasks

  • Docstrings for comb_jit, num_compositions_jit and simplex_grid: say the overflow boundary is np.iinfo(np.intp).max and therefore platform-dependent (2³¹−1 on wasm32)
  • Document the wasm32 limits in the JupyterLite user-facing notes: moved grid-size boundary, int32 default integer, 2–4 GB memory cap shared with the whole kernel (coordinate with #931)
  • Grep sweep of remaining intp arithmetic sites — starting with k_array_rank_jit (quantecon/util/combinatorics.py:103), which calls comb_jit and so shares the same np.iinfo(np.intp).max boundary — to confirm none has a non-size/index use where int64 would be the better contract (expected outcome: none)

Related

  • #942 — proposed widening comb_jit to int64; superseded by the analysis above and to be closed unmerged. Its body still opens with "Fixes #929", so merging it would both reverse the keep-intp decision and auto-close this issue.
  • #943 (lazy Numba compilation for #930) — stacked on #942: its commit list includes #942's 97518e0, and its own commit replaces the eager signature with lazy @jit(nopython=True, cache=True) plus np.int64() casts, so merging it as-is would land the int64 widening (tests, docstrings and the test_simplex_grid_comb_int64 smoke test attributed to this issue). It must be rebased onto main with 97518e0 dropped and intp retained (np.iinfo(np.intp).max, no widening casts) before merge.
  • #944 — simplex_grid fails in the browser with RuntimeError: no compiled object yet for comb_jit. The cause is a cache-hit/cache-miss linking bug in the emscripten-forge Numba build, reported upstream as emscripten-forge/recipes#6309: patch 0006's _serialize_wasm_linking_libraries calls _get_compiled_object() on a cache=True callee restored from the persistent cache, whose object _object_getbuffer_hook has already consumed. It triggers when a cache-miss cache=True caller links a cache-hit cache=True callee in the same warm session, so it needs a warm cache and does not reproduce in a cold single session; comb_jit's eager signature is not the cause (it only makes comb_jit the deterministic trigger). This, not integer width, is what currently blocks simplex_grid in the browser, and the same cached→cached pattern exists in k_array_rank_jit → comb_jit and _lemke_howson_tbl → _pivoting / _lex_min_ratio_test.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the docstrings for comb_jit and num_compositions_jit in quantecon/util/numba.py, simplex_grid in quantecon/_gridtools.py, and k_array_rank_jit in quantecon/util/combinatorics.py. Review the JupyterLite user-facing notes and grep remaining intp arithmetic sites. Done means the platform-dependent limits and result-dtype behavior are documented and the audit finds no non-size/index use requiring a different contract.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python, wasm
Domain
documentation, web-dev
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.