emscripten-core / emscripten-core/emscripten

Intensive `fseek`/`fread` calls are slow if main thread proxied to pthread, regardless of the file system used

Open
#27,143 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
27.6k
Forks
3.6k
Avg merge
1d 1h
Merged PRs (30d)
105

Description

**Version of emscripten/emsdk:**
emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 6.0.0 (afa15e0c56d1292e073c2c91bafc1d5e0cdf0dd3)
clang version 23.0.0git (https:/github.com/llvm/llvm-project c6c2ad52586c28e74a724570985a48dc0022d250)
Target: wasm32-unknown-emscripten
Thread model: posix

**Description**

Previous comment on #24639

> It becomes extremely slow when I change IDBFS to WasmFS in my project. According to the flame graph, the bottleneck is located at `FT_Open_Face`. This function performs thousands of `fseek` and `fread` operations.
>
> After some ablation studies, I found that `fread`/`fseek` is slower only in worker. This means that if main thread proxied to pthread, __even with JS FS, read operations will still be very slow__.
>
> I tried JSPI also, but that is even worse, so I manually put a small file in IDBFS, mimicking objectstore structure in JSFS and made a example reading that 250,000 times (16B per time and seek the next)
MCVE: [uihp/mcve-emscripten-wasmfs-slow-small-op](https://github.com/uihp/mcve-emscripten-wasmfs-slow-small-op)
>
> __Result:__
> ```
> PTHREAD=ON & PROXY=OFF & WASMFS=OFF : time elapsed 147ms
> PTHREAD=ON & PROXY=ON & WASMFS=OFF : time elapsed 14427ms
> PTHREAD=ON & PROXY=ON & WASMFS=ON : timeout>2min (OPFS) / 25717ms (fetchfs)
> PTHREAD=OFF & WASMFS=OFF : time elapsed 61ms
> PTHREAD=OFF & WASMFS=ON (JSPI) : timeout>2min (OPFS)
> ```
>
> The results show that IDBFS becomes significantly slower after main thread being proxied to pthread, and WasmFS is even slower. Without using fetchfs based on JSFileFS (I think this is likely memfs in wasm), it even reaches a speed that is almost unusable.

I came across a solution involving OPFS, but I'm not sure if it's relevant to this.
It's been inactive for a few months, and this is actually unrelated to WasmFS mentioned in the original issue, so I open this separate issue.

In additional, this is the part of flame graph for example code:

Image

Where, the time taken for each `__orginal_main` is about 70ms

Contributor guide

Open the contributing guide

Research direction

Start by running the linked MCVE and comparing its PTHREAD, PROXY, WASMFS, and filesystem configurations. Profile the repeated fseek/fread path around FT_Open_Face and the main-thread proxy boundary. Done means the slowdown is explained and a reproducible benchmark shows the affected configuration no longer has the reported severe regression.

Written by the indexing model from the issue text.

Assessment

Tech stack
wasm
Domain
operating-systems, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.