Research scalable browser-backed storage for Kandelo VFS mounts

## Why

Kandelo’s virtual filesystem (VFS) gives WebAssembly programs a POSIX-style filesystem. In a normal browser boot, the root image
and writable scratch mounts use MemoryFileSystem, so their live contents are backed by browser memory.

That works well for current demos, but it creates a practical ceiling: the useful filesystem must fit within the browser’s memory
budget. A growing WordPress site is a concrete example. Its database, plugins, themes, caches, and uploaded media could eventually
become too large to keep resident at once.

Kandelo already has a mount router and a low-level OpfsFileSystem
(https://github.com/Automattic/kandelo/blob/main/host/src/vfs/opfs.ts) implementation. However, normal browser boots do not
currently use it for their root or scratch mounts, and the existing backend has important browser and POSIX boundaries. There was
also an earlier whole-image OPFS snapshot prototype
(https://github.com/Automattic/kandelo/blob/c424cb2135dcf70ede5005ceaaa780d27bc2a10d/docs/plans/2026-05-11-browser-vfs-persistence-opfs-prototype.md),
but restoring the complete image into memory addresses persistence rather than filesystem scalability.

“Browser-backed” is more accurate than “disk-backed.” Browser storage APIs do not promise a particular physical storage medium or
native on-disk layout. Storage quotas are implementation-defined, and persistent storage must be requested rather than assumed
(File System Standard (https://fs.spec.whatwg.org/), Storage Standard (https://storage.spec.whatwg.org/)).

## Overall goal

Explore, document, and prototype ways for Kandelo to mount a filesystem whose complete contents do not need to remain in memory.

One possible shape is:

Kandelo programs
↓ POSIX filesystem operations
VFS mount with a bounded in-memory cache
↓
Browser-managed backing store containing the full data

The cache might hold recently accessed file data and metadata up to a configured limit, while a browser storage layer remains the
source of truth. This is only one model to investigate, not a chosen design.

The result should preserve accurate filesystem state and POSIX behavior wherever possible. Browser limitations should be
documented or returned as honest failures rather than hidden behind demo-specific behavior.

## Research directions

Ideas worth comparing include:

- Extending the existing Origin Private File System (OPFS) backend. This could map guest files directly to OPFS entries or store
Kandelo-managed blocks, metadata, and a journal inside one or more container files.

- Representing directories, file metadata, and fixed-size data chunks in IndexedDB (https://w3c.github.io/IndexedDB/). Its
transactions may help with consistency, but transaction contention and data-copying costs need measurement.

- Combining IndexedDB metadata with OPFS file data. This may offer useful tradeoffs, but there is no single transaction spanning
both stores, so interrupted updates and recovery would need careful study.

- Offering a user-selected directory through the File System Access API (https://wicg.github.io/file-system-access/) where
supported. This would be an explicit, permission-based mount with additional questions around revoked permission and files
changed by other applications.

- Layering an immutable or lazily loaded base filesystem with a persistent writable overlay.
- Comparing page-, block-, chunk-, and whole-file caches, including read-through, write-through, and delayed write-back
approaches.

- Exploring journals, snapshots, content-addressed chunks, or other recovery mechanisms without assuming that any one is required.

An approach that is ruled out with good evidence is still a useful research result.

## Questions to answer

- How would each design preserve file identity, rename behavior, files that remain open after unlink, concurrent descriptors,
locks, symbolic links, hard links, modes, ownership, timestamps, and mapped files?

- What should write, close, and fsync promise? What survives a worker crash, browser termination, page reload, or partially
completed update?

- How can memory use remain bounded when there are many small files, a few very large files, dirty cached data, or repeated random
access?

- What happens when the browser denies persistence, changes its quota estimate, evicts best-effort storage, or reports
QuotaExceededError?

- How should multiple Kandelo processes, workers, tabs, or machines using the same origin coordinate access?
- Which behavior works across Chromium, Firefox, and WebKit? Where a browser lacks a required capability, can Kandelo provide an
equivalent fallback, or should the mount fail explicitly?

- What permission and recovery experience would a user-selected directory require?
- How would a new backend fit the existing FileSystemBackend and VirtualPlatformIO mount path without moving filesystem behavior
into a demo?

- Which browser-specific differences are unavoidable, and which POSIX-visible behavior should remain consistent with Kandelo’s
Node.js host?

## Useful experiments and evidence

A useful investigation could produce:

- A comparison of candidate APIs, browser support, permission requirements, quota behavior, durability, performance
characteristics, and POSIX gaps.

- One or more narrow prototypes mounted through Kandelo’s normal VFS path.
- Workloads whose stored data is several times larger than the configured memory cache.
- Cold-cache, warm-cache, and cache-thrashing measurements.
- Large sequential imports, random reads and writes, and metadata-heavy operations over many small files.
- A representative large WordPress workload containing database changes and uploaded media.
- Recovery tests that interrupt writes, terminate workers, reload the page, or exhaust available quota.
- Measurements of peak memory, steady-state memory, operation latency, throughput, storage use, and cache behavior.
- Results that record the browser and version, hardware, dataset, cache limit, and persistence state.
- A recommendation describing the most promising architecture, rejected alternatives, known limitations, and possible follow-up
implementation work.

The prototype does not need to outperform MemoryFileSystem for every workload. The important questions are whether it can bound
memory use, preserve correct behavior, and provide understandable performance and durability tradeoffs.

## Scope note

This is intentionally ambitious, exploratory work and is not urgent. A production-ready replacement for every browser mount is not
required as the first outcome. Research, reproducible measurements, and a focused prototype are enough to guide smaller follow-up
issues.

Related current documentation: Kandelo filesystem architecture
(https://github.com/Automattic/kandelo/blob/main/docs/architecture.md#filesystem) and browser filesystem support
(https://github.com/Automattic/kandelo/blob/main/docs/browser-support.md#filesystem).