emscripten-core / emscripten-core/emscripten

`MAXIMUM_MEMORY` is capped at 16 GB for wasm64 builds

Open
#25,862 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
27.6k
Forks
3.6k
Avg merge
1d 1h
Merged PRs (30d)
105

Description

**Version of emscripten/emsdk:**

```
emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 4.0.20 (6913738ec5371a88c4af5a80db0ab42bad3de681)
clang version 22.0.0git (https:/github.com/llvm/llvm-project d163988dd2833f28fbca8c144265108d25ae7bd2)
Target: wasm32-unknown-emscripten
Thread model: posix
InstalledDir: /home/luna/Software/emsdk/emsdk/upstream/bin
```

**Failing command line in full:**

(Copied from CMake logs, sorry about the length)

```
/emsdk/upstream/bin/wasm-ld -o scran.wasm -lembind-mt-rtti CMakeFiles/scran_wasm.dir/src/NumericMatrix.cpp.o CMakeFiles/scran_wasm.dir/src/cbind.cpp.o CMakeFiles/scran_wasm.dir/src/subset.cpp.o CMakeFiles/scran_wasm.dir/src/delayed.cpp.o CMakeFiles/scran_wasm.dir/src/matrix_stats.cpp.o CMakeFiles/scran_wasm.dir/src/initialize_from_arrays.cpp.o CMakeFiles/scran_wasm.dir/src/initialize_from_rds.cpp.o CMakeFiles/scran_wasm.dir/src/initialize_from_mtx.cpp.o CMakeFiles/scran_wasm.dir/src/initialize_from_hdf5.cpp.o CMakeFiles/scran_wasm.dir/src/transpose_matrix.cpp.o CMakeFiles/scran_wasm.dir/src/rds_utils.cpp.o CMakeFiles/scran_wasm.dir/src/hdf5_utils.cpp.o CMakeFiles/scran_wasm.dir/src/write_sparse_matrix_to_hdf5.cpp.o CMakeFiles/scran_wasm.dir/src/quality_control_rna.cpp.o CMakeFiles/scran_wasm.dir/src/quality_control_adt.cpp.o CMakeFiles/scran_wasm.dir/src/quality_control_crispr.cpp.o CMakeFiles/scran_wasm.dir/src/normalize_counts.cpp.o CMakeFiles/scran_wasm.dir/src/compute_clrm1_factors.cpp.o CMakeFiles/scran_wasm.dir/src/model_gene_variances.cpp.o CMakeFiles/scran_wasm.dir/src/run_pca.cpp.o CMakeFiles/scran_wasm.dir/src/mnn_correct.cpp.o CMakeFiles/scran_wasm.dir/src/scale_by_neighbors.cpp.o CMakeFiles/scran_wasm.dir/src/NeighborIndex.cpp.o CMakeFiles/scran_wasm.dir/src/run_tsne.cpp.o CMakeFiles/scran_wasm.dir/src/run_umap.cpp.o CMakeFiles/scran_wasm.dir/src/build_snn_graph.cpp.o CMakeFiles/scran_wasm.dir/src/cluster_graph.cpp.o CMakeFiles/scran_wasm.dir/src/cluster_kmeans.cpp.o CMakeFiles/scran_wasm.dir/src/score_markers.cpp.o CMakeFiles/scran_wasm.dir/src/run_singlepp.cpp.o CMakeFiles/scran_wasm.dir/src/score_gsdecon.cpp.o CMakeFiles/scran_wasm.dir/src/hypergeometric_test.cpp.o CMakeFiles/scran_wasm.dir/src/aggregate_across_cells.cpp.o CMakeFiles/scran_wasm.dir/src/get_error_message.cpp.o /__w/scran.js/scran.js/extern/installed/lib/libigraph.a /__w/scran.js/scran.js/extern/installed/lib/libhdf5.a /__w/scran.js/scran.js/extern/installed/lib/libhdf5_cpp.a /__w/scran.js/scran.js/extern/installed/lib/libigraph.a /__w/scran.js/scran.js/extern/installed/lib/libhdf5.a /emsdk/upstream/emscripten/cache/sysroot/lib/wasm64-emscripten/libz.a -L/emsdk/upstream/emscripten/cache/sysroot/lib/wasm64-emscripten -L/emsdk/upstream/emscripten/src/lib /emsdk/upstream/emscripten/cache/sysroot/lib/wasm64-emscripten/libz.a /emsdk/upstream/emscripten/cache/sysroot/lib/wasm64-emscripten/crtbegin.o -lGL-mt-getprocaddr -lal -lhtml5 -lstubs -lnoexit -lc-mt -ldlmalloc-mt -lcompiler_rt-mt -lc++-mt-noexcept -lc++abi-mt-noexcept -lsockets-mt -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -mwasm64 /tmp/tmp3f3gbm9xlibemscripten_js_symbols.so --import-memory --shared-memory --strip-debug --export=malloc --export=free --export=_emscripten_stack_alloc --export=__getTypeName --export=_embind_initialize_bindings --export=_emscripten_thread_free_data --export=_emscripten_thread_crashed --export=__wasm_call_ctors --export=_emscripten_tls_init --export=_emscripten_thread_init --export=emscripten_stack_get_current --export=_emscripten_stack_restore --export=emscripten_stack_set_limits --export=_emscripten_thread_exit --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-if-defined=__start_em_lib_deps --export-if-defined=__stop_em_lib_deps --export-if-defined=__start_em_js --export-if-defined=__stop_em_js --export-table -z stack-size=2097152 --max-memory=53687091200 --initial-memory=16777216 --no-entry --no-stack-first --table-base=1 --global-base=1024
```

This gives:

```
wasm-ld: error: maximum memory too large, cannot be greater than 17179869184
```

Is this restriction really necessary for wasm64? Maybe browsers have some memory limit per page, but if I'm running Node on the backend, I should be free to use as much memory as my machine has available.

Currently I've been asking for 50GB, this is a scientific application that might process very large datasets. But honestly, it would be even better to have a `-sMAXIMUM_MEMORY=inf` option, which would allow me to re-use the same Wasm binary on HPCs with even more RAM.

FWIW I already compile a separate Wasm binary for web applications, so I can easily set a lower `MAXIMUM_MEMORY` in the web configuration to respect any browser limits.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.