new feature: add capability-rooted secure filesystem access
- Dominant language
- Rust
- Stars
- 5.4k
- Forks
- 825
- Avg merge
- 1d 14m
- Merged PRs (30d)
- 127
Description
## Feature Description
Add a local filesystem implementation that can safely accept untrusted OpenDAL object paths while confining every filesystem operation to a server-controlled, pre-opened directory capability.
GreptimeDB currently maintains a [`SecureFsBackend`](https://github.com/GreptimeTeam/greptimedb/pull/8708) built on `cap_std::fs::Dir` for SQL features such as `COPY FROM/TO`, `COPY DATABASE`, and external file tables. We would like to upstream the reusable part so other OpenDAL users do not need to maintain their own security-sensitive filesystem backend.
This would provide a stronger and explicitly documented contract than `services::Fs`. The current OpenDAL threat model states that `Fs` prevents lexical root escape but follows normal host filesystem symlink semantics, so it is not a sandbox when untrusted users can influence paths or descendants under the configured root.
The required security properties are:
1. Filesystem access begins from an opened directory capability, instead of repeatedly joining an untrusted path to an ambient host path.
2. Parent components, absolute paths, root components, and platform path prefixes are rejected.
3. Symlink resolution must never escape the capability root. In-root symlinks may follow the semantics guaranteed by the selected capability library.
4. Path replacement or rename races must not turn a checked path into ambient access outside the root.
5. Every advertised path-bearing operation must preserve the same confinement boundary.
6. Deleting the capability root itself is rejected.
7. Unsupported operations are not advertised and return `Unsupported`.
## Problem and Solution
### Why lexical validation around `services::Fs` is insufficient
A layer that validates a path string and then delegates to `services::Fs` would still rely on ambient path resolution inside `Fs`. A symlink under the configured root, or a path changed between validation and use, could invalidate the earlier check. Canonicalizing before each operation would have the same check-then-use problem and would not safely handle creation of paths that do not yet exist.
`cap_std::fs::Dir` provides the needed primitive. Operations are resolved relative to an opened directory handle, and attempts to escape that directory are denied. Once a file has been opened through the capability API, it can be converted to a Tokio file for asynchronous reads and writes without resolving the untrusted path again.
### Public API decision
In the [OpenDAL 0.58 migration discussion](https://github.com/GreptimeTeam/greptimedb/pull/8742#issuecomment-5202606316), @Xuanwo suggested that a secure layer may be the most reusable abstraction. After reviewing the current OpenDAL composition API, I see three possible shapes:
1. **Dedicated `SecureFs` service (my current recommendation).** Add an optional `opendal-service-secure-fs` crate. Its builder could accept either a trusted ambient root path, opened once during construction, or a pre-opened `cap_std::fs::Dir`. This directly models a component that performs capability-rooted filesystem operations and can advertise only the operations it implements securely.
2. **`SecureFsLayer` that replaces filesystem operations.** The layer would take a capability root and return a service that handles filesystem operations instead of forwarding them. This keeps the requested layer name, but `Layer::apply_service` receives an erased `Servicer`, cannot return an error, and normally wraps the inner service. It cannot cleanly enforce that the wrapped service is `Fs`, and silently replacing an arbitrary inner service could be surprising.
3. **An opt-in secure mode in the existing `Fs` service.** This avoids a second filesystem service, but it puts two different security contracts in the same implementation and complicates optional dependency boundaries.
I would like maintainer guidance on whether “layer” is a strict API requirement or whether a dedicated service better fits the current service/layer model.
### Proposed first-version scope
The initial implementation would support:
- `create_dir` through the capability root.
- `stat` for the root and descendants.
- Full and range reads, opening files through the capability before asynchronous I/O.
- Empty, append, multi-buffer, and `if_not_exists` writes.
- Lazy, bounded-batch listing, including missing paths and entries removed during iteration.
- Non-recursive and recursive deletion of descendants, while rejecting deletion of the capability root.
The initial version would leave `copy`, `rename`, `compose`, `restore`, `presign`, and writer abort unsupported and unadvertised. Copy and rename could be added later using same-root capability operations once their OpenDAL semantics and race behavior are covered.
### Runtime handling
`cap_std` filesystem methods are synchronous. GreptimeDB currently sends them to its project-specific blocking pool, which cannot be reused upstream. OpenDAL's existing `Fs` service already uses Tokio and `tokio::task::spawn_blocking` for blocking file operations, so the narrowest first implementation would use the same runtime pattern.
`OperationContext::executor` schedules futures rather than blocking closures, so using it directly would still block an async worker. Adding a general blocking-executor abstraction seems broader than this feature. Is a Tokio-backed implementation acceptable for the first version?
### Test and threat-model coverage
The contribution would include:
- Parent traversal, absolute path, and platform prefix rejection.
- Windows drive, UNC, verbatim path, and separator-confusion coverage on Windows.
- Descendant symlinks targeting files and directories outside the capability root.
- Permitted in-root symlink behavior, matching `cap_std`'s documented contract.
- Concurrent path-component replacement or rename attempts.
- Sandbox-root deletion rejection.
- Root and descendant stat, full and range reads, write modes, parent creation, and exact error mapping.
- Lazy listing batches, missing and non-directory paths, and entries removed during iteration.
- Recursive and non-recursive deletion, including missing targets.
- Exact capability reporting and `Unsupported` behavior.
- Linux and Windows CI coverage.
- An explicit capability-rooted security contract in `SECURITY-THREAT-MODEL.md`, while leaving the existing `services::Fs` symlink contract unchanged.
### Rollout
GreptimeDB would keep its current backend until the upstream implementation is released. A separate GreptimeDB integration change would then run the existing security, COPY, and file-engine tests against the upstream implementation and remove the local backend only after equivalent or stronger behavior is demonstrated.
Related GreptimeDB tracking issue: https://github.com/GreptimeTeam/greptimedb/issues/8988
## Additional Context
- GreptimeDB introduced the current backend after multiple security reports involving SQL-triggered local filesystem access.
- No existing OpenDAL issue matching `secure filesystem` or `sandbox fs symlink` was found before preparing this proposal.
- I am willing to implement the accepted design and its Linux and Windows test coverage.
The main questions for maintainers are:
1. Should this be a dedicated service, a service-replacing layer, or an opt-in secure mode in `Fs`?
2. Should the public constructor accept a pre-opened `cap_std::fs::Dir`, a trusted root path, or both?
3. Is a Tokio-backed implementation acceptable for the first version?
4. Should copy and rename remain unsupported initially?
Contributor guide
Research direction
Start by reviewing the existing services::Fs implementation and the current OpenDAL service/layer composition API. Resolve whether the accepted design is a dedicated service, a service-replacing layer, or an opt-in mode, along with constructor and runtime choices. Done includes the listed capability-boundary operations, security and threat-model coverage in SECURITY-THREAT-MODEL.md, exact capability reporting, and Linux and Windows tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100