cloudflare / cloudflare/workers-rs

[Feature] Honor exact http_body::Body::size_hint via FixedLengthStream

Open
#1,051 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
3.7k
Forks
429
Avg merge
20h 28m
Merged PRs (30d)
7

Description

### Is there an existing issue for this?

- [x] I have searched the existing issues

Related: #511 (http return types always use chunked transfer encoding). This request is the streaming half of that: keep the live body, set `Content-Length` when the size is known exactly.

### Description

`worker::http::response::to_wasm` always wraps a non-empty `http_body::Body` in a plain `ReadableStream`:

```rust
let readable_stream = if body.is_end_stream() {
None
} else {
let stream = BodyStream::new(body);
Some(wasm_streams::ReadableStream::from_stream(stream).into_raw())
};

Ok(web_sys::Response::new_with_opt_readable_stream_and_init(
readable_stream.as_ref(),
&init.into(),
)?)
```

workerd then emits `Transfer-Encoding: chunked` and does not send `Content-Length`. Setting `Content-Length` on the Rust / JS headers is ignored; the runtime derives framing from the body type. Per [the Workers streams docs](https://developers.cloudflare.com/workers/runtime-apis/streams/transformstream/#fixedlengthstream), only a `FixedLengthStream`, string, or `TypedArray` gets `Content-Length`. A short completed chunked body is indistinguishable from a full one for clients that treat EOF as success.

`http_body::Body::size_hint()` can already be exact (`SizeHint::with_exact(n)`). Axum uses that to set `Content-Length`. `to_wasm` never looks at it.

In #511 the suggested workaround was to poll once and buffer when `size_hint` has an upper bound. Please do not do that. An upper bound is best-effort, and collecting is unusable for multi-MiB streaming responses.

### Ask

In `to_wasm`, if `body.size_hint().exact()` is `Some(n)`, wrap the existing `ReadableStream` in [`FixedLengthStream(n)`](https://developers.cloudflare.com/workers/runtime-apis/streams/transformstream/#fixedlengthstream) before constructing `web_sys::Response`. Do not collect the body.

That is an identity transform: chunks still stream, workerd sets `Content-Length: n`, and the stream errors if too many or too few bytes are written. `worker` already exposes `FixedLengthStream` for R2 puts.

Use `exact()` only. Do not treat a lone `upper` bound as known length.

### Repro

`worker` 0.7.5, features `http` + `axum`. Return an `http::Response` whose body implements `http_body::Body` with `SizeHint::with_exact(n)` (or any Axum body that already advertises an exact hint). Observed on a deployed Worker:

- status `200`
- `Transfer-Encoding: chunked`
- no `Content-Length`

Same Rust body in front of Axum (non-Worker) gets `Content-Length`.

Contributor guide

Open the contributing guide

Research direction

Start at worker::http::response::to_wasm and inspect how the existing BodyStream becomes a wasm_streams ReadableStream, then review the worker FixedLengthStream exposure used for R2 puts. When Body::size_hint().exact() is Some(n), the existing chunks should remain streamed through FixedLengthStream(n), with the resulting response reporting Content-Length and errors for too few or too many bytes; upper-only hints must remain unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, wasm
Domain
api, backend
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.