emscripten-core / emscripten-core/emscripten
ptrToIdx() and idxToPtr() macros to unify 2GB, 4GB and Wasm64 programming
- Dominant language
- C++
- Stars
- 27.6k
- Forks
- 3.6k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 105
Description
In the old PR https://github.com/emscripten-core/emscripten/pull/19289 I proposed three helper functions `{{{ ptrToIdx() }}}`, `{{{ convertPtrToIdx() }}}` and `{{{ idxToPtr() }}}` to provide a unified programming model for converting between Wasm pointers and pre-shifted typed array heap indices.
There was some pushback in the PR, so the feature did not land.
I ended up landing the feature into Unity's fork of Emscripten, relegating to the idea that I would make those a Unity .js library `{{{ macro }}}`s.
But later on I realize that it is not actually possible for Emscripten users to generate "system level" JS library wide `{{{ }}}` macros, the parsing order is a PITA to deal with.
So I would like to gauge the temperature of whether we might revisit landing `{{{ ptrToIdx() }}}`, `{{{ convertPtrToIdx() }}}` and `{{{ idxToPtr() }}}` as end-user programmable macros into Emscripten?
Given that Wasm produces pointers as 64-bit BigInts, or int32s that need to be `>>> 0`d in 4GB mode, it would be ideal to have one programming model to cover all of these cases.
Currently Emscripten has ad hoc `#if CAN_ADDRESS_2GB` handling in some call sites:
https://github.com/emscripten-core/emscripten/blob/6ca370067cacb6a6102ed7245c611660ab38a514/src/lib/libstrings.js#L118-L120
and ad hoc `#if MEMORY64` handling in others:
https://github.com/emscripten-core/emscripten/blob/6ca370067cacb6a6102ed7245c611660ab38a514/src/lib/libccall.js#L54-L56
along with `to64()` and `from64()` macros that perform pointer<->heap index conversion in Wasm64 mode only.
The automatic `foo__sig: '....'` model helps automate this challenge in some cases, but it has [undesirable interactions with JSPI](https://github.com/juj/wasm_webgpu/blob/54c51d715d68f22a62e1f84eb072737e8a8270c0/lib/lib_webgpu.js#L672), and it results in pessimization of generated code size.
In the [juj/wasm_webgpu](https://github.com/juj/wasm_webgpu) library, I ended up needing the following constructs for most convenient handling of 2GB+4GB+Wasm64: https://github.com/juj/wasm_webgpu/blob/54c51d715d68f22a62e1f84eb072737e8a8270c0/lib/lib_webgpu.js#L26-L88
(where the functions are a bit awkwardly named to avoid possible name conflict that I was anticipating with #19289)
Would there be interest in revisiting the idea of adding `{{{ ptrToIdx() }}}`, `{{{ convertPtrToIdx() }}}` and `{{{ idxToPtr() }}}` macros? I would like to be able to provide a single documented API to Unity users so that they can manage 2GB, 4GB and Wasm64 programming models in a unified fashion.
Contributor guide
Assessment
This issue has not been assessed yet.