emscripten-core / emscripten-core/emscripten

sqlite CRUD operations are getting slower over time in Chrome when ASYNCIFY is used and the database is on a IDBFS mount

Open
#26,332 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
27.6k
Forks
3.6k
Avg merge
1d 1h
Merged PRs (30d)
105

Description

All of our SQL tests with sqlite are running slower and slower over time in our ASYNCIFY'd environment on a IDBFS mount...only in Chrome.
FF and Safari are performing with roughly the same times each loop.
Even deleting the DB file in the IDBFS doesn't help, I can revert to the initial loop time only when deleting the indexed DB content.
Running on MEMFS I can't observe this behaviour. Also running without ASYNCIFY I can't observe this behavior..(unfortunately we need both IDBFS and ASYNCIFY).
I created a little GitHub project exposing the behaviour at [https://github.com/leopatras/em_slow_sqlite_chrome](https://github.com/leopatras/em_slow_sqlite_chrome).
Attached here as a zip archive

[em_slow_sqlite_chrome_issue.zip](https://github.com/user-attachments/files/25522723/em_slow_sqlite_chrome_issue.zip)

Some simple insert/delete queries are running slower in each iteration even if the db file is deleted between each iteration. Only deleting the whole indexed DB path for the storage does bring back the initial time measured.
Profiling shows that operations are getting slower regardless of the type of operation(inserts are getting slower, deletes are getting slower), so something in IDBFS together with ASYNCIFY seems to cause the slowness.
Generally when looking at a performance recording lot of pauses are visible in the recording , the actual work done appears smaller than the pauses where the browser does nothing (see attached screenshot)

Image

**Version of emscripten/emsdk:**
emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 5.0.0 (a7c5deabd7c88ba1c38ebe988112256775f944c6)
clang version 23.0.0git (https:/github.com/llvm/llvm-project 358db292cc6a9a8a5448a296f643312289f328d7)
Target: wasm32-unknown-emscripten
Thread model: posix
InstalledDir: /Users/leo/w/github/emsdk/upstream/bin

**Full link command and output with `-v` appended:**
```
emcc \
-O2 \
-s EXPORT_ALL=1 \
-s EXIT_RUNTIME=1 \
-s ASSERTIONS=1 \
-s FORCE_FILESYSTEM=1 \
-s ASYNCIFY \
-lidbfs.js \
--profiling \
--emrun \
-v \
-o main.html main.o sqlite3.o
/Users/leo/w/github/emsdk/upstream/bin/clang --version
/Users/leo/w/github/emsdk/upstream/bin/wasm-ld -o main.wasm /var/folders/fv/7l4yvm9d2bvbsvl80kmhp8b40000gn/T/tmp21hvav9nlibemscripten_js_symbols.so --export=emscripten_stack_get_end --export=emscripten_stack_get_free --export=emscripten_stack_get_base --export=emscripten_stack_get_current --export=emscripten_stack_init --export=_emscripten_stack_alloc --export=__funcs_on_exit --export=__wasm_call_ctors --export=_emscripten_stack_restore --export=strerror --export=malloc --export=free --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-if-defined=__start_em_lib_deps --export-if-defined=__stop_em_lib_deps --export-if-defined=__start_em_js --export-if-defined=__stop_em_js --export-if-defined=main --export-if-defined=__main_argc_argv --export-if-defined=fflush --export-table -z stack-size=65536 --no-growable-memory --initial-heap=16777216 --no-entry --no-stack-first --table-base=1 --global-base=1024 main.o sqlite3.o -L/Users/leo/w/github/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten -L/Users/leo/w/github/emsdk/upstream/emscripten/src/lib -lGL-getprocaddr -lal -lhtml5 -lstubs-debug -lc-debug -ldlmalloc-debug -lcompiler_rt -lc++-debug-noexcept -lc++abi-debug-noexcept -lsockets -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr
/Users/leo/w/github/emsdk/upstream/bin/llvm-objcopy main.wasm main.wasm '--remove-section=.debug*' --remove-section=producers
/Users/leo/w/github/emsdk/upstream/bin/wasm-emscripten-finalize -g --bigint --no-legalize-javascript-ffi main.wasm -o main.wasm --detect-features
/Users/leo/w/github/emsdk/node/20.18.0_64bit/bin/node /Users/leo/w/github/emsdk/upstream/emscripten/tools/compiler.mjs -
/Users/leo/w/github/emsdk/upstream/bin/wasm-opt --strip-target-features --post-emscripten -O2 --low-memory-unused --asyncify --pass-arg=asyncify-asserts --pass-arg=asyncify-propagate-addlist '--pass-arg=asyncify-imports@env.invoke_*,env.__asyncjs__*,*.fd_sync,*.emscripten_promise_await,*.__syscall_poll,*.emscripten_idb_load,*.emscripten_idb_store,*.emscripten_idb_delete,*.emscripten_idb_exists,*.emscripten_idb_clear,*.emscripten_sleep,*.emscripten_wget_data,*.emscripten_scan_registers,*._load_secondary_module,*.emscripten_fiber_swap' --zero-filled-memory --pass-arg=directize-initial-contents-immutable main.wasm -o main.wasm -g --mvp-features --enable-bulk-memory --enable-bulk-memory-opt --enable-call-indirect-overlong --enable-multivalue --enable-mutable-globals --enable-nontrapping-float-to-int --enable-reference-types --enable-sign-ext
/Users/leo/w/github/emsdk/node/20.18.0_64bit/bin/node /Users/leo/w/github/emsdk/upstream/emscripten/tools/acorn-optimizer.mjs /var/folders/fv/7l4yvm9d2bvbsvl80kmhp8b40000gn/T/emscripten_temp_r2gnon2o/main.js JSDCE --closure-friendly -o /var/folders/fv/7l4yvm9d2bvbsvl80kmhp8b40000gn/T/emscripten_temp_r2gnon2o/main.jso1.js
/Users/leo/w/github/emsdk/node/20.18.0_64bit/bin/node /Users/leo/w/github/emsdk/upstream/emscripten/tools/preprocessor.mjs - shell.html
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.