Nullus157 / Nullus157/async-compression

Encoder does not support intermediate flushes before the reader has ended

Open
#154 11 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
666
Forks
118
Avg merge
17h 18m
Merged PRs (30d)
9

Description

*disclaimer: this is about fixing https://github.com/apollographql/router/issues/1572, for which I'm under tight timing constraints, so right now this issue's goal (and the related PR) is discussing and finding a quick fix that we can use directly in the router, and then I'll help find a proper solution.*

I'm encountering an issue with the following setup:
- I use axum with tower-http's `CompressionLayer`, which uses async-compression
- the router sends a HTTP response with multipart
- one multipart element is sent right away, the next one is sent after 5s
- if the response is compressed, the router will wait for 5s to send both parts at the same time, while without compression, the first part comes immediately, and the second one after 5s

I tracked that down to async-compression, where in the tokio based `Encoder`, due to the use of `ready!()`, whenever the underlying reader returns `Poll::Pending`, it is transmitted directly to the caller, so there is no flush until the entire data has been read:

https://github.com/Nemo157/async-compression/blob/ada65c660bcea83dc6a0c3d6149e5fbcd039f739/src/tokio/bufread/generic/encoder.rs#L63-L74

I would like the encoder to send the data it already compressed when the reader returned `Poll::Pending`, and let the upper layer decide on buffering or sending the HTTP chunk directly.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.