feat(services): Add `wasi-fs` backend using `wasi:filesystem` interface
- Dominant language
- Rust
- Stars
- 5.4k
- Forks
- 825
- Avg merge
- 1d 14m
- Merged PRs (30d)
- 127
Description
### Summary
Add a new OpenDAL backend that leverages the [`wasi:filesystem`](https://github.com/WebAssembly/wasi-filesystem) interface for file operations. This enables OpenDAL to access the host filesystem when running as a WebAssembly component, with the host runtime (e.g., wasmtime, wasmer) providing the actual filesystem implementation.
### Motivation
Currently, the `services-fs` backend cannot compile to `wasm32-wasip2` because it depends on tokio's filesystem features, which are not supported on WASM targets.
However, WASM components can access the host filesystem through the standardized `wasi:filesystem` interface. A `wasi-fs` backend would:
1. **Enable filesystem access in WASM components** — Components could read/write files preopened by the host runtime
2. **Complement existing backends** — `memory` and `s3` work on WASM, but there's no local filesystem option
3. **Leverage existing WASI ecosystem** — Runtimes like wasmtime already implement `wasi:filesystem`
### Proposed Implementation
A new `services-wasi-fs` feature/backend that:
- **Imports** `wasi:filesystem/types` and `wasi:filesystem/preopens` interfaces
- **Uses** `wasi:io/streams` for read/write operations
- **Works with** preopened directories provided by the host
- **Compiles to** `wasm32-wasip2` target only (or conditional compilation)
### Use Case
We're building WASM components that expose storage capabilities. We want to:
1. Create an OpenDAL-based storage component
2. Export `wasi:filesystem` interface for downstream consumers
3. Support multiple backends (memory, S3, **and host filesystem**)
Without a `wasi-fs` backend, components cannot access files that the host runtime preopens for them.
### Research Notes
| Backend | WASM-compatible | Notes |
|---------|-----------------|-------|
| `services-memory` | ✅ | Works out of the box |
| `services-s3` | ✅ | Works with reqwest/reqsign |
| `services-fs` | ❌ | Requires tokio fs (not WASM-compatible) |
| `services-wasi-fs` | 🆕 | **Proposed** — uses `wasi:filesystem` imports |
### Related
- WASI filesystem spec: https://github.com/WebAssembly/wasi-filesystem
- Preview 2 (Component Model): https://github.com/WebAssembly/WASI/blob/main/preview2/README.md
Contributor guide
Research direction
Start by reviewing the existing services-fs backend and the linked wasi:filesystem and Preview 2 specifications. Verify how wasi:filesystem/types, preopens, and wasi:io/streams map to the proposed backend, then build for wasm32-wasip2. Done means a services-wasi-fs backend can access host-preopened files while compiling for the target.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, wasm
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100