bytecodealliance / bytecodealliance/wasmtime

Proposal: first-class Instance::reset() for pooled reuse of untrusted guests

Open
#14,313 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
18.6k
Forks
1.8k
Avg merge
1d 19h
Merged PRs (30d)
121

Description

#### Feature

We embed wasmtime in a high-throughput HTTP proxy (the embedding from #14312) and pool WASI 0.2 component instances across requests, resetting each instance to pristine state between checkouts. Today we do this from the embedder side via two small introspection accessors we carry as a local patch. That patch is ~90 additive lines with tests. We are working through our company's open source approval process to be able to offer it upstream, and will link the PR here once we can. Either way it is the minimal primitive, not the complete solution: this issue proposes the runtime-owned reset that the accessors cannot deliver.

The feature: a way to reset a live instance back to its just-instantiated state, for example `component::Instance::reset(&self, store) -> Result<()>`, and possibly a core `Instance::reset` as well. Reset rewinds linear memories, tables, mutable globals, and internal flags (data/elem drops, lazy funcref init) so the instance is indistinguishable from a fresh instantiation of the same `InstancePre` in the same store, without re-running `start` or component initializers.

#### Benefit

It makes instance reuse safe for untrusted guests, which is currently a gap. `wasmtime serve` reuses instances by trusting the guest to keep itself request-ready (#9542), default-on for WASIp3 but deliberately default-off for WASIp2 (`max_instance_reuse_count` defaults to 1). Embedders whose p2 guests are untrusted or operator-supplied cannot opt into trust-based reuse at all, and per-request instantiation is a host-wide kernel ceiling, not a per-core cost.

The three strategies below all give every request a pristine instance. They differ in who restores the state and whether the kernel is involved:

* **fresh instantiate (on-demand)**: new `Store` + instantiate per request on the default allocator. Each instance mmaps its linear memory and unmaps it on drop, so every request pays kernel address-space work that serializes on the process mmap_lock.
* **fresh instantiate (pooling allocator)**: same per-request instantiate, but instances come from the pooling allocator's pre-reserved slab (no per-instance mmap). Recycling a slot decommits its dirty pages back to the CoW image with madvise, which costs TLB-shootdown IPIs across the process's CPUs.
* **reset-based reuse (embedder-side)**: instantiate once, then per request: call, then reset in userspace. The reset memcmps each 4 KiB page against a post-instantiate snapshot, copies back only the pages that differ, and rewinds mutable globals via the accessors. No syscalls at all.

Measured on a dual-socket Xeon Gold 6330 (2x28 cores, 112 threads with SMT), dispatch-bound component call, aggregate calls/s:

| threads | fresh instantiate (on-demand) | fresh instantiate (pooling allocator) | reset-based reuse (embedder-side) |
|--------:|------------------------------:|--------------------------------------:|----------------------------------:|
| 1 | 26.0K | 36.7K | 143.9K |
| 8 | 26.3K | 92.8K | 971.3K |
| 96 | 21.3K | 191.1K | 4.32M |

On-demand instantiation is flat from 1 to 96 threads, with per-call latency inflating from 0.026 ms to 2.88 ms as threads queue on the kernel. The pooling allocator is better but stalls near 191K/s: only 2x more throughput for 12x more threads past 8. The userspace reset scales with cores: 22.6x the best instantiate-per-request strategy at 96 threads. Per-call isolation overhead at 1 thread: 25.2 us (on-demand) and 17.1 us (pooling) vs 5.8 us (reset).

A runtime-owned reset would beat our embedder-side numbers further and close a correctness gap at the same time:

* The runtime owns the memory images (`MemoryImageSlot`) and the PAGEMAP_SCAN machinery, so discovery can be O(dirty pages) with no embedder snapshot. Our embedder compare pass is O(total memory): about 2.5 us at 256 KiB but 230 us at 16 MiB, where PAGEMAP_SCAN stays at 2 to 8 us for small dirty counts.
* The runtime can reach state no public API can: table mutations (`table.set`, `elem.drop` effects), data/element drop flags, additional linear memories, and component-layer runtime state. Our embedder reset compensates with a load-time self-check and by constraining accepted modules. A first-class reset deletes that class of edge entirely.
* It could make p2 reuse in `wasmtime serve` safe by default rather than trust-gated.

#### Implementation

Semantics: rewind to the post-instantiate point, as above. The runtime already holds everything needed: the memory image as the pristine reference, initial global values and element segments from the module, and the internal flags.

Two mechanism notes from our production experience that a design should account for:

* Restoring page contents is not restoring page state. A memcpy restore re-writes the page, so it stays an anonymous WRITTEN copy and every subsequent reset re-copies it. Per-reset cost ratchets toward O(cumulative-touched) for both PAGEMAP_SCAN and compare discovery (at 4 MiB: 2.0 us/reset at 8 touched pages, 161.7 us at all 1024). Only decommit (madvise back to the image) re-arms clean kernel state.
* Decommit has a concurrency cost microbenchmarks miss: a madvise-based reset collapsed for us at ~256 concurrent connections on the same 112-thread host (TLB-shootdown IPI storms). A reset API should be able to choose per page between memcpy and decommit, and expose a shootdown-free mode. Periodic full recycle stays structurally necessary regardless (as `wasmtime serve` already ships via `max_instance_reuse_count`).

There are also aspects of the runtime we are not familiar enough with to know how they would interact with a reset: fuel and epochs, async and concurrent state, resource handles, resetting while other instances share the store, and whether the pooling allocator could service a reset without unsharing images. We defer to maintainers on all of those. Happy to share the full benchmark methodology and an embedder's-eye API review now, and to contribute the accessors patch and implementation work under maintainer direction once our approval process completes.

#### Alternatives

* **What we run today: embedder-side reset via two introspection accessors** (`Instance::core_instances`, `Instance::defined_globals`, ~90 additive lines, PR pending our company's open source approval). Snapshot post-instantiate state, then compare-and-copy linear memory and rewind mutable globals on checkout. Proven in production and the source of the numbers above, but this approach has a hard ceiling no accessor extension can lift: discovery is O(total memory) per reset (kernel dirty-page tracking does not help an embedder, whose own memcpy restores keep pages marked written forever, per the ratchet note above), and tables, drop flags, extra memories, and component-layer state stay out of reach. The O(dirty) column and the completeness both require the runtime-owned reset. The accessors are still worth landing on their own: they are the minimal primitive that works today and remain useful plumbing under any future reset.
* **Memory-only `Memory::reset_to_image(&self, store)`**: a much smaller design surface that exposes the existing PAGEMAP_SCAN plus image-slot machinery. Delivers the O(dirty) column, and globals are already coverable from the embedder side with the accessors, leaving tables and drop flags as the residual gap.
* **Fresh instantiate per request**: the safe default today, but the table above shows it is a host-wide ceiling (21K to 191K calls/s regardless of core count) that binds at proxy request rates.
* **Trust-based reuse (#9542 temporal isolation)**: works for trusted p3 guests, unusable when guests are untrusted or operator-supplied, which is exactly the p2 population.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.