cloudflare / cloudflare/workers-rs
[Feature] Honor exact http_body::Body::size_hint via FixedLengthStream
- Dominant language
- Rust
- Stars
- 3.7k
- Forks
- 429
- Avg merge
- 20h 28m
- Merged PRs (30d)
- 7
Description
### Is there an existing issue for this?
- [x] I have searched the existing issues
Related: #511 (http return types always use chunked transfer encoding). This request is the streaming half of that: keep the live body, set `Content-Length` when the size is known exactly.
### Description
`worker::http::response::to_wasm` always wraps a non-empty `http_body::Body` in a plain `ReadableStream`:
```rust
let readable_stream = if body.is_end_stream() {
None
} else {
let stream = BodyStream::new(body);
Some(wasm_streams::ReadableStream::from_stream(stream).into_raw())
};
Ok(web_sys::Response::new_with_opt_readable_stream_and_init(
readable_stream.as_ref(),
&init.into(),
)?)
```
workerd then emits `Transfer-Encoding: chunked` and does not send `Content-Length`. Setting `Content-Length` on the Rust / JS headers is ignored; the runtime derives framing from the body type. Per [the Workers streams docs](https://developers.cloudflare.com/workers/runtime-apis/streams/transformstream/#fixedlengthstream), only a `FixedLengthStream`, string, or `TypedArray` gets `Content-Length`. A short completed chunked body is indistinguishable from a full one for clients that treat EOF as success.
`http_body::Body::size_hint()` can already be exact (`SizeHint::with_exact(n)`). Axum uses that to set `Content-Length`. `to_wasm` never looks at it.
In #511 the suggested workaround was to poll once and buffer when `size_hint` has an upper bound. Please do not do that. An upper bound is best-effort, and collecting is unusable for multi-MiB streaming responses.
### Ask
In `to_wasm`, if `body.size_hint().exact()` is `Some(n)`, wrap the existing `ReadableStream` in [`FixedLengthStream(n)`](https://developers.cloudflare.com/workers/runtime-apis/streams/transformstream/#fixedlengthstream) before constructing `web_sys::Response`. Do not collect the body.
That is an identity transform: chunks still stream, workerd sets `Content-Length: n`, and the stream errors if too many or too few bytes are written. `worker` already exposes `FixedLengthStream` for R2 puts.
Use `exact()` only. Do not treat a lone `upper` bound as known length.
### Repro
`worker` 0.7.5, features `http` + `axum`. Return an `http::Response` whose body implements `http_body::Body` with `SizeHint::with_exact(n)` (or any Axum body that already advertises an exact hint). Observed on a deployed Worker:
- status `200`
- `Transfer-Encoding: chunked`
- no `Content-Length`
Same Rust body in front of Axum (non-Worker) gets `Content-Length`.
Contributor guide
Research direction
Start at worker::http::response::to_wasm and inspect how the existing BodyStream becomes a wasm_streams ReadableStream, then review the worker FixedLengthStream exposure used for R2 puts. When Body::size_hint().exact() is Some(n), the existing chunks should remain streamed through FixedLengthStream(n), with the resulting response reporting Content-Length and errors for too few or too many bytes; upper-only hints must remain unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, wasm
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100