emscripten-core / emscripten-core/emscripten
`ABORTING_MALLOC=1` do not work with `ALLOW_MEMORY_GROWTH=1` and `MAXIMUM_MEMORY=4GB`
- Dominant language
- C++
- Stars
- 27.6k
- Forks
- 3.6k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 105
Description
**Version of emscripten/emsdk:**
4.0.6-git
I would like to use all available memory before having to change the pointer size to 8 bytes (`MEMORY64`) hence `MAXIMUM_MEMORY=4GB`. It seems that in this mode `malloc` could fail silently even though the `ABORTING_MALLOC=1` is set. I encountered this bug while working on the `STANDALONE_WASM` project where the memory grow mechanism works a little differently, I not sure how related failed malloc is to memory grow implementation, but I was able to narrow down the problem to the following example:
Project layout:
├ library.cpp
└ playground.js
```cpp
// library.cpp
#include
extern "C" void *mallocInCpp(size_t size) {
return malloc(size);
}
```
```js
// playground.js
Module.onRuntimeInitialized = () => {
console.log('chunk 1', Module._malloc((512 * 1024 * 1024)));
console.log('chunk 2', Module._malloc((512 * 1024 * 1024)));
console.log('chunk 3', Module._malloc((512 * 1024 * 1024)));
console.log('chunk 4', Module._malloc((512 * 1024 * 1024)));
console.log('chunk 5', Module._malloc((512 * 1024 * 1024)));
console.log('chunk 6', Module._malloc((512 * 1024 * 1024)));
console.log('chunk 7', Module._malloc((512 * 1024 * 1024)));
console.log('chunk 8', Module._malloc((512 * 1024 * 1024)));
};
```
To build standard 2GB version:
```shell
emcc library.cpp -o out2gb.js --post-js playground.js --no-entry \
-sASSERTIONS=2 -sEXPORTED_FUNCTIONS=[_malloc,_mallocInCpp] \
-sALLOW_MEMORY_GROWTH=1 -sABORTING_MALLOC=1
```
To build 4GB version:
```shell
emcc library.cpp -o out4gb.js --post-js playground.js --no-entry \
-sASSERTIONS=2 -sEXPORTED_FUNCTIONS=[_malloc,_mallocInCpp] \
-sALLOW_MEMORY_GROWTH=1 -sABORTING_MALLOC=1 -sMAXIMUM_MEMORY=4GB
```
Then the output of `out2gb.js` is as expected:
```
// node ./out2gb.js
Heap resize call from 16908288 to 537001984 took 0.7890999999999977 msecs. Success: true
chunk 1 68456
Heap resize call from 537001984 to 1073872896 took 1.6869999999999976 msecs. Success: true
chunk 2 536939376
Heap resize call from 1073872896 to 1610743808 took 2.590600000000002 msecs. Success: true
chunk 3 1073810296
Cannot enlarge memory, requested 2147553280 bytes, but the limit is 2147483648 bytes!
RuntimeError: Aborted(Cannot enlarge memory arrays to size 2147553280 bytes (OOM). If you want malloc to return NULL (0) instead of this abort, do not link with -sABORTING_MALLOC (that is, the default when growth is enabled is to not abort, but you have overridden that))
```
But the output of `out4gb.js` is not as expected, failed malloc silently returns `0`:
```
// node ./out4gb.js
Heap resize call from 16777216 to 537001984 took 0.8843999999999994 msecs. Success: true
chunk 1 68456
Heap resize call from 537001984 to 1073872896 took 1.8088000000000015 msecs. Success: true
chunk 2 536939376
Heap resize call from 1073872896 to 1610743808 took 3.1115999999999993 msecs. Success: true
chunk 3 1073810296
Heap resize call from 1610743808 to 2147614720 took 4.724999999999998 msecs. Success: true
chunk 4 1610681216
Heap resize call from 2147614720 to 2684485632 took 5.6026000000000025 msecs. Success: true
chunk 5 2147552136
Heap resize call from 2684485632 to 3221422080 took 5.890000000000001 msecs. Success: true
chunk 6 2684423056
Heap resize call from 3221422080 to 3858890752 took 7.761199999999995 msecs. Success: true
chunk 7 3221293976
chunk 8 0
```
A js side `malloc` wrapper could fix that:
```js
const originalMalloc = Module._malloc;
Module._malloc = (size) => {
const ptr = originalMalloc(size);
if (ptr) return ptr;
throw new Error('malloc failed');
}
```
but that doesn't help with the `malloc` calls that failed on the cpp/wasm side.
---
I tried a few more different values for `MAXIMUM_MEMORY`:
- 4294901760 (4GB - 64KB wasm page size) - malloc silent fail
- 4278190080 (4GB - 16MB initial memory) - malloc silent fail
- 4194304000 (4GB - 96MB grow cap) - malloc silent fail
- 3758096384 (3.5GB) - interestingly malloc is working as expected, error is thrown
---
EDIT: other interesting behavior for `MAXIMUM_MEMORY=4GB`:
```js
console.log('ptr', Module._malloc((4 * 1024 * 1024 * 1024) - (68 * 1024))); // 4GB - 68KB -> 0 (silent fail)
console.log('ptr', Module._malloc((4 * 1024 * 1024 * 1024) - (69 * 1024))); // 4GB - 69KB -> RuntimeError
```
Contributor guide
Assessment
This issue has not been assessed yet.