bytecodealliance / bytecodealliance/wasmtime

wasi-http(p3): Consider exposing a knob to control outgoing body buffering

Open
#14,160 0 comments 0 reactions 0 assignees View on GitHub
wasi-http wasi:impl
Dominant language
Rust
Stars
18.6k
Forks
1.8k
Avg merge
1d 19h
Merged PRs (30d)
121

Description

Currently the http Preview 3 implementation does not provide a way of controlling the outgoing body buffering, which results in keeping the host/guest in lock-step, potentially affecting components that perform multiple writes to the body.

Compare the p3 implementation at https://github.com/bytecodealliance/wasmtime/blob/main/crates/wasi-http/src/p3/body.rs#L333, with the p2 implementation at https://github.com/bytecodealliance/wasmtime/blob/main/crates/wasi-http/src/p2/body.rs#L417

Consider the `p3_http_echo.rs` test program currently in-tree:

```rust

use {
test_programs::p3::{
service::exports::wasi::http::handler::Guest as Handler,
wasi::http::types::{ErrorCode, Request, Response},
wit_future, wit_stream,
},
wit_bindgen::StreamResult,
};

struct Component;

test_programs::p3::service::export!(Component);

impl Handler for Component {
/// Return a response which echoes the request headers, body, and trailers.
async fn handle(request: Request) -> Result {
let headers = request.get_headers();
let (_, result_rx) = wit_future::new(|| Ok(()));
let (body, trailers) = Request::consume_body(request, result_rx);

let (response, _result) = if headers
.get("x-host-to-host")
.into_iter()
.any(|v| v == b"true")
{
// This is the easy and efficient way to do it...
Response::new(headers, Some(body), trailers)
} else {
// ...but we do it the more difficult, less efficient way here to exercise various component model
// features (e.g. `future`s, `stream`s, and post-return asynchronous execution):
let (trailers_tx, trailers_rx) = wit_future::new(|| todo!());
let (mut pipe_tx, pipe_rx) = wit_stream::new();

wit_bindgen::spawn_local(async move {
let mut body_rx = body;
let mut chunk = Vec::with_capacity(1024);
loop {
let (status, buf) = body_rx.read(chunk).await;
chunk = buf;
match status {
StreamResult::Complete(_) => {
chunk = pipe_tx.write_all(chunk).await;
assert!(chunk.is_empty());
}
StreamResult::Dropped => break,
StreamResult::Cancelled => unreachable!(),
}
}

drop(pipe_tx);

trailers_tx.write(trailers.await).await.unwrap();
});

Response::new(headers, Some(pipe_rx), trailers_rx)
};

Ok(response)
}
}

// Unused function; required since this file is built as a `bin`:
fn main() {}
```

Running the component through `wasmtime serve`:

```shell
./target/release/wasmtime serve -Wcomponent-model-async -Sp3,cli -O pooling-allocator --addr=127.0.0.1:8093
```

and benchmarking with [`oha`](https://github.com/hatoo/oha) for RPS:

```shell
oha --no-tui -z 20s -c 1 -m POST -D /tmp/body.bin http://127.0.0.1:8093/

Summary:
Success rate:100.00%
Total:20000.5816 ms
Slowest:17.1687 ms
Fastest:6.6237 ms
Average:8.0752 ms
Requests/sec:123.7464 <--
```

Re-running the component, _assuming a knob to configure the outgoing body buffering, similar to preview 2's `http-outgoing-body-buffer-chunks` option_:

```shell
./target/release/wasmtime serve -Wcomponent-model-async -Sp3,cli -O pooling-allocator -Shttp-p3-outgoing-body-buffer-chunks=8 --addr=127.0.0.1:8093
```

Re-running the RPS benchmark:

```shell
oha --no-tui -z 20s -c 1 -m POST -D /tmp/body.bin http://127.0.0.1:8093/
Success rate:100.00%
Total:20001.4951 ms
Slowest:4.4801 ms
Fastest:1.5558 ms
Average:2.1880 ms
Requests/sec:456.0159 <--
```

This represents ~3.7x improvement locally; the improvement is obviously variable depending on the number of chunks chosen.

As far as I can tell there is no existing way to influence this, however, it is possible that I am missing some context on why this was not added in the initial implementation and if that is the case, I can close this issue.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.