emscripten-core / emscripten-core/emscripten

Embind modules vs no modules

Open
#13,495 1 comment 5 reactions 0 assignees View on GitHub
wontfix
Dominant language
C++
Stars
27.6k
Forks
3.6k
Avg merge
1d 1h
Merged PRs (30d)
105

Description

I'm working on a library called [OpenCascade.js](https://github.com/donalffons/opencascade.js), which aims at exposing the OpenCascade CAD library to JavaScript. The library provides an API for doing complex geometric operations in 3D on the web, powered by Emscripten :slightly_smiling_face:. There are already some projects using it.

I would like to ask for some advice regarding 3 issues, I'm having:
1. The startup-time of the finished Emscripten module is very long
2. The build-time tends to be extremely long
3. Using a modularized approach (see below), I get a file size increase by more than 200%

# Background

The build system runs in two phases:
1. It analyzes the entire code bases using clang and auto-generates a total of approximately 180k (!) lines of Embind-code
2. Emscripten takes the Embind-Code and generates the JS+WASM "binaries"

Since OpenCascade is a large library, I have investigated the potential of splitting up the entire library into modules, for two reasons:
1. Instead of loading the entire library, a developer could just cherry-pick the relevant modules for a specific app, saving on file size (and potentially reducing startup-time, depending on the use case)
2. The build time of the non-modularized version of the library is very long.

All builds are made using `-O3`optimizations and `AGGRESSIVE_VARIABLE_ELIMINATION`. The modularized version consists of 1 `MAIN_MODULE` and 4 `SIDE_MODULE`s.

Here are my results, comparing the modularized version against the non-modularized version.

# Metrics

## Build Time

**Non-Modularized** : Takes roughly 15:30 (!!) hours on my fairly modern laptop. That's not including the time to generate the Embind-Code. The build is running single-threaded 99% of the time.
**Modularized**: Takes less than 2:00h (probably less than 1:30h), with the builds running in parallel

## Startup Time

Vertical axis: Startup time in milliseconds
Horizontal axis: Attempt number (with caching enabled)
Tested in Chrome stable

![](https://chart.googleapis.com/chart?cht=lc&chs=400x240&chxt=x%2Cy&chxl=0%3A%7C01%7C02%7C03%7C04%7C05%7C06%7C07%7C08%7C09%7C10&chdlp=r&chdl=Modularized%7CNot%20Modularized&chco=3399CC%2C80C65A&chxr=1%2C0%2C16896&chd=e%3A.Y1T1W0w1V0i0t1L2D0W%2C..j.f0f4gRbTZ3alcSbL)

It seems like Chrome does a good job of caching the non-modularized version of the library, but doesn't really seem to cache any of the `SIDE_MODULE`s.

I use this code for loading the `SIDE_MODULE`s (the sequence of loading is important):
```js
for(let lib of libs) {
await oc.loadDynamicLibrary(lib, {loadAsync: true, global: true, nodelete: true, allowUndefined: false});
}
```

## File Size

**Non-Modularized**
```js
// ==========================================
// 222K opencascade.full.js
// 43M opencascade.full.wasm [exports] : 33
// ==========================================
// 43M total
// ==========================================
```

**Modularized**
```js
// ==========================================
// 25M opencascade.core.js
// 21M opencascade.core.wasm [exports] : 69768
// 22M opencascade.dataExchangeBase.wasm [exports] : 68556
// 15M opencascade.dataExchangeExtra.wasm [exports] : 38328
// 33M opencascade.modelingAlgorithms.wasm [exports] : 75720
// 19M opencascade.visualApp.wasm [exports] : 56596
// ==========================================
// 135M total
// ==========================================
```

Number of exports as reported by `wasm-opt -n file.wasm --metrics`.

# Conclusion / Question
Overall, the Non-Modularized version is much faster and smaller. However, the insanely long build times make it a bit of a nightmare to maintain. The build time of the Modularized version on the other hand is perfectly acceptable in my opinion.

Does anyone have recommendations on what could be done to find a better balance of build time, startup time and file size?

(Trying to keep this as short as possible. There is more detailed information in [this discussion](https://github.com/donalffons/opencascade.js/discussions/27)).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.