emscripten-core / emscripten-core/emscripten
Fetch API with EMSCRIPTEN_FETCH_PERSIST_FILE failed to save to indexeddb when PROXY_TO_PTHREAD is set
- Dominant language
- C++
- Stars
- 27.6k
- Forks
- 3.6k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 109
Description
Code to reproduce the issue:
```
#include
#include
#include
#include
#include
#include
int main()
{
emscripten_fetch_attr_t attr;
emscripten_fetch_attr_init(&attr);
strcpy(attr.requestMethod, "GET");
attr.attributes = EMSCRIPTEN_FETCH_LOAD_TO_MEMORY | EMSCRIPTEN_FETCH_PERSIST_FILE | EMSCRIPTEN_FETCH_APPEND | EMSCRIPTEN_FETCH_SYNCHRONOUS;
emscripten_fetch_t *fetch = nullptr;
static std::mutex s_mutex;
{
std::lock_guard guard(s_mutex);
fetch = emscripten_fetch(&attr, "myfile.dat");
}
printf("Finished downloading %llu bytes from URL %s. status %d.\n", fetch->numBytes, fetch->url, fetch->status);
emscripten_fetch_close(fetch);
emscripten_fetch_attr_init(&attr);
strcpy(attr.requestMethod, "GET");
attr.attributes = EMSCRIPTEN_FETCH_NO_DOWNLOAD | EMSCRIPTEN_FETCH_SYNCHRONOUS | EMSCRIPTEN_FETCH_LOAD_TO_MEMORY;
fetch = emscripten_fetch(&attr, "myfile.dat");
assert(fetch->status == 200 && "emscripten_fetch from IndexedDB should have found the file");
emscripten_fetch_close(fetch);
return 0;
}
```
Compile command:
```
python emcc.py temp.c -s FETCH -s WASM -s USE_PTHREADS -s PROXY_TO_PTHREAD -s FORCE_FILESYSTEM -s EXIT_RUNTIME=1 -s ASYNCIFY=1 -s FETCH_SUPPORT_INDEXEDDB=1 -s FETCH_DEBUG -x c++ -std=c++17 -o temp.html
```
I'm using emscripten 3.1.17
It's wired that the fetch command succeeded in downloading the file but failed to save the file to indexeddb. Checking the storage information page for indexeddb on Chrome DevTools, you can find that an indexeddb database is created for the test site but with empty data.
With `-s FETCH_DEBUG` command, the page produces the following log:
```
fetch: indexedDB.open(dbname="emscripten_filesystem", dbversion="1");
temp.js:4320 fetch: IndexedDB successfully opened.
temp.js:4322 fetch: IndexedDB :.[object IDBDatabase]
temp.js:4569 fetch: IndexedDB not available!
fetchLoadCachedData @ temp.js:4569
_emscripten_start_fetch @ temp.js:4771
imports. @ temp.js:5399
$func36 @ temp.wasm:0x28db
$func27 @ temp.wasm:0xede
$func34 @ temp.wasm:0x1e34
$dynCall_ii @ temp.wasm:0x22a0b
ret. @ temp.js:5433
(anonymous) @ temp.js:1134
(anonymous) @ temp.js:3999
invokeEntryPoint @ temp.js:3999
self.onmessage @ temp.worker.js:150
temp.js:4760 fetch: starting (cached) XHR: IndexedDB not available!
performCachedXhr @ temp.js:4760
fetchLoadCachedData @ temp.js:4570
_emscripten_start_fetch @ temp.js:4771
imports. @ temp.js:5399
$func36 @ temp.wasm:0x28db
$func27 @ temp.wasm:0xede
$func34 @ temp.wasm:0x1e34
$dynCall_ii @ temp.wasm:0x22a0b
ret. @ temp.js:5433
(anonymous) @ temp.js:1134
(anonymous) @ temp.js:3999
invokeEntryPoint @ temp.js:3999
self.onmessage @ temp.worker.js:150
temp.js:4378 fetch: xhr.timeout: 0, xhr.withCredentials: false
temp.js:4379 fetch: xhr.open(requestMethod="GET", url: "myfile.dat", userName: undefined, password: undefined);
temp.js:4495 fetch: xhr.send(data=null)
temp.js:4725 fetch: ready state change. e: [object Event]
temp.js:4419 fetch: allocating 1296 bytes in Emscripten heap for xhr data
temp.js:4443 fetch: xhr of URL "myfile.dat" / responseURL "http://localhost:8000/myfile.dat" succeeded with status 200
temp.js:4738 fetch: operation success. Caching result.. e: [object ProgressEvent]
temp.js:4755 fetchCacheData :.undefined
temp.js:4530 fetch: IndexedDB not available!
fetchCacheData @ temp.js:4530
cacheResultAndReportSuccess @ temp.js:4756
xhr.onload @ temp.js:4444
fetchXHR @ temp.js:4497
performCachedXhr @ temp.js:4761
fetchLoadCachedData @ temp.js:4570
_emscripten_start_fetch @ temp.js:4771
imports. @ temp.js:5399
$func36 @ temp.wasm:0x28db
$func27 @ temp.wasm:0xede
$func34 @ temp.wasm:0x1e34
$dynCall_ii @ temp.wasm:0x22a0b
ret. @ temp.js:5433
(anonymous) @ temp.js:1134
(anonymous) @ temp.js:3999
invokeEntryPoint @ temp.js:3999
self.onmessage @ temp.worker.js:150
load (async)
fetchXHR @ temp.js:4429
performCachedXhr @ temp.js:4761
fetchLoadCachedData @ temp.js:4570
_emscripten_start_fetch @ temp.js:4771
imports. @ temp.js:5399
$func36 @ temp.wasm:0x28db
$func27 @ temp.wasm:0xede
$func34 @ temp.wasm:0x1e34
$dynCall_ii @ temp.wasm:0x22a0b
ret. @ temp.js:5433
(anonymous) @ temp.js:1134
(anonymous) @ temp.js:3999
invokeEntryPoint @ temp.js:3999
self.onmessage @ temp.worker.js:150
temp.js:4748 fetch: IndexedDB store failed.
storeError @ temp.js:4748
fetchCacheData @ temp.js:4531
cacheResultAndReportSuccess @ temp.js:4756
xhr.onload @ temp.js:4444
fetchXHR @ temp.js:4497
performCachedXhr @ temp.js:4761
fetchLoadCachedData @ temp.js:4570
_emscripten_start_fetch @ temp.js:4771
imports. @ temp.js:5399
$func36 @ temp.wasm:0x28db
$func27 @ temp.wasm:0xede
$func34 @ temp.wasm:0x1e34
$dynCall_ii @ temp.wasm:0x22a0b
ret. @ temp.js:5433
(anonymous) @ temp.js:1134
(anonymous) @ temp.js:3999
invokeEntryPoint @ temp.js:3999
self.onmessage @ temp.worker.js:150
load (async)
fetchXHR @ temp.js:4429
performCachedXhr @ temp.js:4761
fetchLoadCachedData @ temp.js:4570
_emscripten_start_fetch @ temp.js:4771
imports. @ temp.js:5399
$func36 @ temp.wasm:0x28db
$func27 @ temp.wasm:0xede
$func34 @ temp.wasm:0x1e34
$dynCall_ii @ temp.wasm:0x22a0b
ret. @ temp.js:5433
(anonymous) @ temp.js:1134
(anonymous) @ temp.js:3999
invokeEntryPoint @ temp.js:3999
self.onmessage @ temp.worker.js:150
temp.html:1237 Finished downloading 1296 bytes from URL myfile.dat. status 200.
temp.js:4152 fetch: Deleting id:0 of [object XMLHttpRequest]
temp.js:4569 fetch: IndexedDB not available!
fetchLoadCachedData @ temp.js:4569
_emscripten_start_fetch @ temp.js:4771
imports. @ temp.js:5399
$func36 @ temp.wasm:0x28db
$func27 @ temp.wasm:0x11f0
$func34 @ temp.wasm:0x1e34
$dynCall_ii @ temp.wasm:0x22a0b
ret. @ temp.js:5433
(anonymous) @ temp.js:1134
(anonymous) @ temp.js:3999
invokeEntryPoint @ temp.js:3999
self.onmessage @ temp.worker.js:150
temp.js:4716 fetch: operation failed: IndexedDB not available!
reportError @ temp.js:4716
fetchLoadCachedData @ temp.js:4570
_emscripten_start_fetch @ temp.js:4771
imports. @ temp.js:5399
$func36 @ temp.wasm:0x28db
$func27 @ temp.wasm:0x11f0
$func34 @ temp.wasm:0x1e34
$dynCall_ii @ temp.wasm:0x22a0b
ret. @ temp.js:5433
(anonymous) @ temp.js:1134
(anonymous) @ temp.js:3999
invokeEntryPoint @ temp.js:3999
self.onmessage @ temp.worker.js:150
temp.js:1070 Aborted(Assertion failed: fetch->status == 200 && "emscripten_fetch from IndexedDB should have found the file", at: tester.c,36,main)
```
I added the following piece of code to check `Fetch.dbInstance`:
```
#if FETCH_DEBUG
console.log('fetch: IndexedDB :.' + Fetch.dbInstance);
#endif
```
The code is added at line 55(inside onsuccess handle of Fetch.openDatabase) and line 540(before `fetchCacheData`) in Fetch.js , as you can see in the upper log, `Fetch.openDatabase` succeeded in openning a new database, `Fetch.dbInstance` is not null, but after successfully downloaded the file, before `fetchCacheData`, variable `Fetch.dbInstance` turned out to be null.
I searched the whole source of emscripten and found no assignment to `Fetch.dbInstance` other than onsuccess and onerror handle of `Fetch.openDatabase`, so I don't know why `Fetch.dbInstance` can be null.
By the way, I noticed that in `emscripten_fetch.c`, there is a `globalFetchIdCounter`, which is not thread-safe, I'm not sure using a mutex to protect just `emscripten_fetch` like I did in the demo code is enough or should I use this mutex to protect the whole fetch procedure, until I get the return value of fetch ?
Contributor guide
Assessment
This issue has not been assessed yet.