emscripten-core / emscripten-core/emscripten
[library_dylink.js] How to link a Rust side module with C++ main module: missing `invoke_` functions in proxyHandler
- Dominant language
- C++
- Stars
- 27.6k
- Forks
- 3.6k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 105
Description
Please include the following in your bug report:
**Version of emscripten/emsdk:**
emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.68 (ceee49d2ecdab36a3feb85a684f8e5a453dde910)
clang version 20.0.0git (https:/github.com/llvm/llvm-project 5cc64bf60bc04b9315de3c679eb753de4d554a8a)
Target: wasm32-unknown-emscripten
Thread model: posix
InstalledDir: /Users/carlo/emsdk/upstream/bin
**The situation:**
I work on [duckdb-wasm](https://github.com/duckdb/duckdb-wasm), a C++ based codebase then compiled to Wasm via Emscripten.
To allow to extend the surface of the project, we allow, both in native and Wasm, to add code at run-time via extensions.
In the case of Wasm this means that main project comes with a JS and Wasm module, extension itself is a Wasm module, and via Emscripten's implementation of `dlopen` import / exports are remapped dynamically and then execution continues with additional functionality.
This works end to end for C++ extensions, where on performing `dlopen` imports are correctly remapped in the JS layer and stuff works as expected.
Basic is like:
```SQL
SELECT st_area('POLYGON((0 0, 0 1, 1 1, 1 0, 0 0))'::geometry);
Catalog Error: Scalar Function with name "st_area" is not in the catalog, but it exists in the spatial extension.
LOAD spatial;
SELECT st_area('POLYGON((0 0, 0 1, 1 1, 1 0, 0 0))'::geometry);
┌─────────────────────────────────────────────────────────────────┐
│ st_area(CAST('POLYGON((0 0, 0 1, 1 1, 1 0, 0 0))' AS geometry)) │
╞═════════════════════════════════════════════════════════════════╡
│ 1.0 │
└─────────────────────────────────────────────────────────────────┘
```
(live demo at https://shell.duckdb.org/#queries=v0,SELECT-st_area('POLYGON((0-0%2C-0-1%2C-1-1%2C-1-0%2C-0-0))'%3A%3Ageometry)~,LOAD-spatial~,SELECT-st_area('POLYGON((0-0%2C-0-1%2C-1-1%2C-1-0%2C-0-0))'%3A%3Ageometry)~)
Now I am looking to do the same, but using Rust-based code, compiled with cargo with target `wasm32-unknown-emscripten`.
**The problem**
Rust code compiles, but on `dlopen` there are some missing symbols errors that are raised by code that looks like to come from https://github.com/emscripten-core/emscripten/blob/main/src/library_dylink.js#L699 that is like:
```js
var proxyHandler = {
get(stubs, prop) {
switch (prop) {
case "__memory_base":
return memoryBase;
case "__table_base":
return tableBase
}
if (prop in wasmImports && !wasmImports[prop].stub) {
return wasmImports[prop]
}
if (!(prop in stubs)) {
var resolved;
stubs[prop] = (...args) => {
resolved ||= resolveSymbol(prop);
return resolved(...args)
}
}
return stubs[prop]
}
};
```
Error is that `invoke_viii`, or `invoke_vii` or similarly named functions are not present at the JavasScript level.
**The hack**
Adding a conditional like:
```diff
var proxyHandler = {
get(stubs, prop) {
+ if (prop.startsWith("invoke_")) {
+ return createDyncallWrapper(prop.substring(7));
+ }
switch (prop) {
case "__memory_base":
```
solves the problem, and shows that basically the class of `invoke_`-functions are special in the fact that their code can be reconstructed starting from the signature to perform the correct indirect call.
This workaround is very brittle, I am looking for a more proper solution / directions / material on how this can be properly supported.
Contributor guide
Assessment
This issue has not been assessed yet.