emscripten-core / emscripten-core/emscripten

Dynamic loading modules which come with JS glue code

Open
#16,668 10 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
27.6k
Forks
3.6k
Avg merge
1d 1h
Merged PRs (30d)
105

Description

We want to separate out our networking library (libNetwork) into a separate dlopen-able module.

libNetwork provides the following API:
```
// INetworkProvider.hpp

struct Response { int code; const char* data; int length; ...};
using ResponseCallback = void (*)(void*, Response*);
...
class INetworkProvider {
virtual void make_request(const char* url, ResponseCallback callback);
...
};

INetworkProvider* create_network_provider();

void destroy_network_provider(INetworkProvider*);
```

Other libraries (like libMyApplication) uses libNetwork as follows:
Use `INetworkProvider.hpp` from libNetwork repository during compilation
Run the code below to get a concrete NetworkProvider (omitting error-checking code):
```
std::shared_ptr createNetworkProvider() {
auto* library = dlopen("libNetwork.so", RTLD_NOW);
createFn_ = dlsym(library, "create_network_provider");
destroyFn_ = dlsym(library, "destroy_network_provider");
return std::shared_ptr{(*createFn_)(), destroyFn_};
}
```
The above code sits inside `libMyApplication`. Once `libMyApplication` has a pointer to INetworkProvider , it can make network requests.

#### Constraints
1. The API provided by `libNetwork` is pure-C except for use of virtual functions (which is assumed to be stable across emscripten versions for now)
2. `libNetwork` and `libMyApplication` might use different versions of the emscripten toolchain for compilation
3. `libNetwork` has some JS glue code when compiled to wasm which is used to proxy the actual networking calls.
4. `libMyApplication` needs `libNetwork` only in some special cases. Hence, we want to download and use `libNetwork` at runtime only when the application needs to do a network call.

Looking at official emscripten documentation (https://emscripten.org/docs/compiling/Dynamic-Linking.html) it looks like we can only enable dynamic linking with a side module. However, this doesn’t fit well with our use case above for the following reasons:
1. A side module must be pure-wasm without JS glue code
2. In our case, we prefer that the application module and network module each have their own system libraries without sharing (since they can be using different emscripten versions). Looking at documentation, the side module wouldn’t have system libraries included.

#### Possible solutions
##### Ideal
In an ideal world, we can compile both `libNetwork` and `libMyApplication` as main modules, and `libMyApplication` can still dlopen `libNetwork`. However, based on the documentation, that has some unhandled corner cases.

##### Acceptable
Allow side modules to have JS glue code and system library code (for the system libraries being used).
In that case, we can build libNetwork as a side module with JS glue code, and then load that dynamically at runtime from `libMyApplication`. Some mechanism needs to be provided to load the associated JS glue code.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.