digidem / digidem/mbtiles-reader
Support OPFS SyncAccessHandle Pool VFS so browsers without SharedArrayBuffer can open files
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 1
- Forks
- 0
- Avg merge
- 6h 14m
- Merged PRs (30d)
- 3
Description
MBTiles.open(path) opens the database with new sqlite3.oo1.OpfsDb(path, 'r') (index.browser.js:128). That VFS fakes synchronous I/O with SharedArrayBuffer + Atomics and an async proxy worker, so it only works in a cross-origin-isolated page. Where SharedArrayBuffer is missing, sqlite-wasm logs "Ignoring inability to install OPFS sqlite3_vfs: Missing SharedArrayBuffer and/or Atomics" and open() then throws.
In practice that means Safari. Cross-origin isolation there requires COEP require-corp, because WebKit has never shipped credentialless (https://bugs.webkit.org/show_bug.cgi?id=230550), and require-corp blocks any cross-origin subresource that isn't CORS-checked. An app that loads third-party map tiles therefore has to choose between isolation and its tile sources, and loses MBTiles support on Safari either way.
SQLite's OPFS SyncAccessHandle Pool VFS (opfs-sahpool, SQLite 3.43+) avoids the problem entirely: it pre-opens a pool of sync access handles, so it needs neither SharedArrayBuffer nor COOP/COEP, and it works in Safari 16.4+. The bundled @sqlite.org/sqlite-wasm (3.51.2-build8) already exports installOpfsSAHPoolVfs, so this needs no dependency change.
Roughly:
const pool = await sqlite3.installOpfsSAHPoolVfs({ name: 'mbtiles' })
await pool.importDb(name, callbackReturningChunks)
const db = new pool.OpfsSAHPoolDb(name)
importDb accepts a callback that returns chunks, so a file can be imported incrementally rather than buffered whole — worth keeping, since callers open files that can be hundreds of megabytes.
Selecting it automatically when typeof SharedArrayBuffer === 'undefined', and otherwise keeping OpfsDb, would make the browser build work everywhere without callers needing to care. An explicit option would also work.
Costs, as far as I can tell:
Memory should go down, not up. The pool VFS drops the SharedArrayBuffer and the dedicated async proxy worker that the default VFS needs; what it adds is a fixed number of open file handles (default capacity 6, growable via addCapacity/reserveMinimumCapacity), which is negligible. Import streams if given a callback, so there's no need to hold the file in memory. I have not measured either VFS, and I could not find published figures.
Performance should improve. SQLite's own documentation calls it "easily the highest OPFS performance of the options", since reads and writes go straight through a sync access handle instead of round-tripping through Atomics.wait against a proxy worker.
The real costs are structural. Only one connection may be open at a time, and a second tab throws rather than queueing. Files are stored under opaque generated names, so the pool's contents aren't browsable or interoperable with files written through the normal OPFS API — a caller that today copies a file into OPFS and opens it by path would import into the pool instead. And the pool's capacity is persistent across sessions, so it should be managed rather than grown unboundedly.
Affected consumers: digidem/map-downloader (its local .mbtiles basemap is unusable in Safari — being documented as unsupported for now) and gmaclennan/mbtiles-viewer (works today only because it serves require-corp and needs no third-party resources; this would let it drop cross-origin isolation).
Alternatives considered: MBTiles.open(File | ArrayBuffer) already works without OPFS via sqlite3_deserialize, but holds the whole file in wasm memory, which rules out large tilesets; and proxying third-party tiles to satisfy require-corp shifts bandwidth and licensing onto the app.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in index.browser.js at the MBTiles.open path around line 128, then read the bundled sqlite-wasm API for installOpfsSAHPoolVfs, importDb, and OpfsSAHPoolDb. Define how selection and pool lifecycle should work when SharedArrayBuffer is unavailable, while preserving the existing path-based behavior otherwise; done means browser builds can open large MBTiles files without cross-origin isolation, including Safari.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, sqlite
- Domain
- databases, web-dev
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100