CompressionLayer re-polls non-fused streaming bodies after EOF and panics

Open
#732 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
65/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
rust
Domain
api, backend

Research direction

Start at CompressionLayer's WrapBody::poll_frame and BodyIntoStream::poll_frame, then run the supplied streaming gzip reproducer with a non-fused Body::from_stream source. Trace the distinction between underlying EOF and a buffered non-data frame; done means the gzip body emits its footer and completes without polling the source after EOF, while trailers and later frames remain possible.

Written by the indexing model from the issue text.

Description

  • I have looked for existing issues (including closed) about this

Bug Report

Version
tower-http v0.7.0 and v0.7.1 (feature = "compression-gzip")

Reproduced on 0.7.0 and 0.7.1. Source inspection of current main shows the relevant BodyIntoStream::poll_frame fallback is unchanged, so it appears affected as well.

Platform
Linux fedora 7.1.10-200.fc44.x86_64 #1 SMP PREEMPT_DYNAMIC Sun Aug 23 16:15:11 UTC 2026 x86_64 GNU/Linux

Also reproduced on a Fedora x86_64 server (VPS).

Crates

compression (Compression / CompressionLayer, compression-gzip feature)

Description

Gzip-compressing a streaming response body built from Body::from_stream(futures::stream::unfold(..)) panics when the source stream reaches EOF, after the gzip encoder has produced the body bytes.

I tried this code:

// tower-http = { version = "=0.7.0", features = ["compression-gzip"] }
// The same reproducer also fails with version = "=0.7.1".
// futures = "0.3.34"
// axum = "0.8.9"
// http-body = "1.1.0"
// http-body-util = "0.1.5"
// async-compression = "0.4.43"
// tokio = "1.53.1" (features = ["rt-multi-thread", "macros", "sync", "time"])
// tower = "0.5.3"
// bytes = "1.12.1"

use axum::body::{Body, to_bytes};
use axum::http::{Request, Response};
use bytes::Bytes;
use tower::ServiceExt;
use tower_http::compression::Compression;

#[tokio::test]
async fn repro_streaming_gzip_panic() {
    let svc = Compression::new(tower::service_fn(|_req: Request<Body>| async {
        let (tx, rx) = tokio::sync::mpsc::channel::<Result<Bytes, std::io::Error>>(64);
        tokio::spawn(async move {
            let chunk = Bytes::from(vec![b'x'; 64]);
            for _ in 0..100 {
                if tx.send(Ok(chunk.clone())).await.is_err() {
                    return;
                }
            }
            tokio::time::sleep(std::time::Duration::from_millis(15)).await;
            // sender dropped here -> the unfold returns None
        });
        let body = Body::from_stream(futures::stream::unfold(rx, |mut rx| async move {
            rx.recv().await.map(|item| (item, rx))
        }));
        Ok::<_, std::convert::Infallible>(
            Response::builder()
                .header("content-type", "application/json")
                .body(body)
                .unwrap(),
        )
    }));

    let req = Request::builder()
        .header("accept-encoding", "gzip")
        .body(Body::empty())
        .unwrap();
    let resp = svc.oneshot(req).await.unwrap();
    // Panics here:
    let _ = to_bytes(Body::new(resp.into_body()), usize::MAX).await.unwrap();
}

I expected the response body to emit gzip data, then a valid footer, and end cleanly at the HTTP-body level.

Instead, this happened:

thread 'tokio-runtime-worker' panicked at .../futures-util-0.3.34/src/stream/unfold.rs:108:21:
Unfold must not be polled after it returned `Poll::Ready(None)`

Reproduces 100% of the time with this compressed non-fused stream-backed body; buffered bodies are unaffected.

WrapBody::poll_frame polls the underlying body after the gzip encoder reaches EOF in order to forward remaining frames. In this case, that eventually causes BodyIntoStream::poll_frame to re-poll an underlying body that has already returned None.

There is an interface mismatch here: http-body recommends that Body implementations remain pollable after EOF, while StreamBody delegates to an arbitrary Stream, whose contract permits panicking after None. In this reproduction, BodyIntoStream::poll_next has already received None from the underlying body's poll_frame and sets yielded_all_data. However, yielded_all_data also represents the distinct case where a non-data frame was encountered, so it cannot by itself mean HTTP-body EOF. When no non-data frame is buffered, BodyIntoStream::poll_frame then falls through to this.body.poll_frame(cx) after its internal stream returns None, which violates the Body post-EOF contract for the wrapper when the wrapped body is backed by a non-fused Stream. Tracking actual underlying-body exhaustion separately, or otherwise making only the true-EOF path fused, would avoid the re-poll without suppressing trailers or other frames that legitimately follow codec EOF.

In a server, the panic aborts the connection task before the HTTP body can complete normally. In this reproducer, the gzip footer itself has already been emitted, but the transport stream is terminated abnormally; clients report a mid-stream reset (for example, HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2) from curl, or "The socket connection was closed unexpectedly" from Bun fetch).

In production I hit this on SSE because our custom compression predicate allowed text/event-stream; the default tower-http predicate excludes SSE. Observed in production: 1,551 panics with this stack over ~46 hours while compressed SSE responses were enabled; after excluding text/event-stream from compression, no further instances were observed.

Related: #419 / #420 ("SSE buffers under compression", resolved in 0.5.2 by #465, which excludes text/event-stream from the default predicate). The adjacent post-codec trailer and frame handling was later changed in #685 and #712; neither guards this non-fused source-body re-poll.

Dominant language
Rust
Stars
913
Forks
231
Avg merge
1d 20h
Merged PRs (30d)
8

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from tower-rs/tower-http

All issues in tower-rs/tower-http

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.