emscripten-core / emscripten-core/emscripten

LazyFile doesn't work if using Brotli compression for the lazily loaded file

Open
#17,415 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
27.6k
Forks
3.6k
Avg merge
1d 1h
Merged PRs (30d)
105

Description

Please include the following in your bug report:

**Version of emscripten/emsdk:**
emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 2.0.31 (4fcbf0239ccca29771f9044c990b0d34fac6e2e7)
clang version 14.0.0 (https://github.com/llvm/llvm-project 8fa2394bad433558f3083cee158043e2fb66d781)
Target: wasm32-unknown-emscripten
Thread model: posix

I am using FS.createLazyFile to load .wasm files lazily for dynamic linking. I was using gzip compression for the .wasm files and it was working perfect. But if I enable brotli compression, loading of the WASM fails.

The reason for that seems to be that we are handling the case where if a file is gzipped we download the whole file instead of making range requests for it. This can be seen in the comments here: https://github.com/emscripten-core/emscripten/blob/8b4443a87f5eab5dbb9adb690f9ebed0a9da4bd9/src/library_fs.js#L1756

For Brotli, this check doesn't return true and hence we end up making range reuqests and in fact only fetching the partial file (compressed length < uncompressed asset length) and thus tries to work with an invalid file essentially.

We should probably also check for brotli compression or any other kind of compression for that matter and download the whole file? Currently we are checking for gzip here:
https://github.com/emscripten-core/emscripten/blob/8b4443a87f5eab5dbb9adb690f9ebed0a9da4bd9/src/library_fs.js#L1709

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.