rivet-dev / rivet-dev/dynamic-apps
Treat the WASM/WASI build target as cfg(unix) so filesystem tools need no per-tool mode-bit patches
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 1k
- Forks
- 51
- Avg merge
- 6m
- Merged PRs (30d)
- 13
Description
Summary
We build coreutils (ls, stat, chmod, …) for wasm32-wasip1. Because that
target is not cfg(unix), every crate's #[cfg(unix)] filesystem code —
the code that reads real st_mode permission bits — is compiled out, and the
#[cfg(not(unix))] fallback (which fabricates permissions from a single
readonly boolean) is compiled in. To get native-Linux-accurate output we've
had to hand-patch each tool to bypass that fallback and fetch mode bits from the
sidecar via a host_fs.path_mode import (landed in #268 / #269).
This issue tracks the larger question: should we make the WASM build target
present as cfg(unix) (a Linux-like target) so these tools "just work" with no
per-tool patches, instead of maintaining a growing pile of #[cfg(target_os = "wasi")] shims?
Why a non-Linux target breaks filesystem tools
uucore has two implementations of display_permissions, cfg-gated:
#[cfg(unix)] // reads REAL bits
pub fn display_permissions(md, ...) -> String {
display_permissions_unix(md.mode() as mode_t, ...) // st_mode
}
#[cfg(not(unix))] // WASI lands HERE — fabricates
pub fn display_permissions(md, display_file_type) -> String {
let write = if md.permissions().readonly() { '-' } else { 'w' };
format!("{file_type}r{write}xr{write}xr{write}x") // r & x HARDCODED on
}
Upstream root cause (uutils/coreutils, uucore 0.5.0):
https://github.com/uutils/coreutils/blob/main/src/uucore/src/lib/features/fs.rs
Consequences of being cfg(not(unix)):
md.mode()isn't even callable —std::os::unix::fs::MetadataExtdoesn't
exist in the WASIstd.- Rust's own
std::os::wasi::fs::MetadataExtdeliberately exposes
dev/ino/nlink/size/atim/…but nomode(), because WASI preview1's
filestatstruct has no mode field. So the bits are absent at thestdlayer
regardless of cfg. - Net effect:
ls -lprinted-rwxrwxrwx(or-r-xr-xr-xif readonly) for
every file — a uniform string derived from one boolean.
Note this is degradation, not deliberate stripping: the tools have "WASI
support," it's just a lossy fallback nobody wired to a host that has the bits.
Evidence: the per-tool workarounds this forces (all at 8145b07)
Each of these exists only because the target isn't cfg(unix):
ls— inject ahost_fs.path_modeimport +mode_for_pathand swap
display_permissions→display_permissions_unix:
https://github.com/rivet-dev/secure-exec/blob/8145b07ebcd1f6553a417a009091c53d7610a04f/registry/native/patches/crates/uu_ls/0001-wasi-host-fs-mode-display.patch#L16-L74stat:
https://github.com/rivet-dev/secure-exec/blob/8145b07ebcd1f6553a417a009091c53d7610a04f/registry/native/patches/crates/uu_stat/0001-wasi-metadata-compat.patchchmod:
https://github.com/rivet-dev/secure-exec/blob/8145b07ebcd1f6553a417a009091c53d7610a04f/registry/native/patches/crates/uu_chmod/0001-wasi-compat.patch- First-party shims hit the same wall —
whichand the shell builtins each
carry their own#[cfg(target_os = "wasi")]host_fsextern:
https://github.com/rivet-dev/secure-exec/blob/8145b07ebcd1f6553a417a009091c53d7610a04f/registry/native/crates/libs/shims/src/which.rs#L16-L58
https://github.com/rivet-dev/secure-exec/blob/8145b07ebcd1f6553a417a009091c53d7610a04f/registry/native/crates/libs/builtins/src/lib.rs#L14-L27 - And the C side already has the mirror-image fix in wasi-libc (which Rust can't
reach, because Rust std bypasses libcstat()on wasip1):
https://github.com/rivet-dev/secure-exec/blob/8145b07ebcd1f6553a417a009091c53d7610a04f/registry/native/patches/wasi-libc/0016-host-fs-mode-and-chmod.patch
Every new fs-touching Rust tool we add will need another such patch.
Options
A. Force cfg(unix) on the current target (cheap, does NOT work)
Passing --cfg unix via RUSTFLAGS flips the cfg on our crates, but the
#[cfg(unix)] path does use std::os::unix::fs::MetadataExt, which the
precompiled WASI std doesn't contain → unresolved import. And even if it
resolved, the WASI Metadata's underlying filestat has no st_mode to
return. Rejected.
B. Custom Linux-like target spec + -Z build-std (the real "treat it like native")
Define a target derived from wasm32-wasip1 with target-family = ["unix", …]
and recompile std/core so Rust's fs routes through the libc-backed unix fs
backend (calls libc stat(), reads st_mode). Then the wasi-libc mode patch
(0016) flows up into Rust automatically and all #[cfg(unix)] tool code works —
no per-tool patches. Blast radius, however, is large:
- Nightly + build-std on every build; per-Rust-version maintenance of a bespoke
target. cfg(unix)flips the entire dep graph, not just perms — signals, process,
users/groups (getpwuid), termios, mmap, net — much of which calls libc
symbols WASI libc doesn't implement → link/ENOSYSbreakage to chase.- The fs backend (
target_os = "wasi") andMetadataExt(target_family = unix) are gated on different cfgs and don't cleanly compose; effectively a
std fork.
C. Move the runtime to WASI preview2 / wasi:filesystem (the clean long-term fix)
preview2 carries richer metadata than preview1's filestat. Migrating (or
upstreaming a wasi MetadataExt::mode() backed by an extended filestat) would
let these tools read real bits natively and let us delete all the patches —
without pretending to be unix.
Ask / decision needed
Decide between: (B) invest in a Linux-like custom target + build-std so fs tools
need zero patches, vs. (C) target preview2, vs. (status quo) keep adding small
per-tool host_fs patches. Leaning C long-term; B is the "make it native" ask
but has broad blast radius. Capturing so the tradeoff is explicit before the
patch pile grows.
Related PRs: #268 (filesystem native-parity: wasi-libc + sidecar host_fs),
#269 (coreutils stat/chmod/ls real permission bits).
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with uutils/coreutils' src/uucore/src/lib/features/fs.rs and compare the referenced WASI patches under registry/native/patches with the which.rs and builtins entry points. Investigate the target-spec, build-std, and preview2 alternatives described here; done means selecting and documenting one direction with its compatibility and maintenance consequences.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, wasm
- Domain
- build-system, operating-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100