ClickHouse / ClickHouse/clickhouse-rs
Default (LZ4) compression breaks on ClickHouse 26.9+: Lz4Decoder hardcodes magic 0x82 but the server default codec is now ZSTD(3)
- Dominant language
- Rust
- Stars
- 559
- Forks
- 172
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 3
Description
## Description
`clickhouse-rs` with its default feature set uses `Compression::Lz4`, which appends `?compress=1` to every read request (`src/query.rs:258-267`) and pipes the response body through `Lz4Decoder` (`src/response.rs:284-292`). `Lz4Decoder` dispatches on nothing — it asserts the LZ4 native-framing method byte:
```rust
// src/compression/lz4.rs
const LZ4_MAGIC: u8 = 0x82;
...
if magic != LZ4_MAGIC {
return Err(Error::Decompression("incorrect magic number".into()));
}
```
ClickHouse [#108786](https://github.com/ClickHouse/ClickHouse/pull/108786) ("Switch the default compression to ZSTD(3) for table data and network", in `26.9.1.810` and later) changes `CompressionCodecFactory::getDefaultCodec` from `LZ4` to `ZSTD(3)`. The HTTP `compress=1` framed output is one of the direct `getDefaultCodec` users, so on 26.9+ the server frames the response with method byte `0x90` (ZSTD) instead of `0x82`.
Consequence: with the default (`lz4`) feature set, **every read fails** on ClickHouse 26.9+ with:
```
Error: Decompression("incorrect magic number")
```
Per the upstream PR's own changelog the HTTP `compress=1` path has no runtime rollback (not controlled by `compatibility`, per-column `CODEC`, or the server `` config), so this can't be worked around server-side.
Workarounds on the client side:
- `Client::with_compression(Compression::None)` — loses compression entirely.
- `Client::with_compression(Compression::Zstd(_))` with the `zstd` feature — this path is **not** affected, because it uses HTTP-level compression (`enable_http_compression=1` + `Accept-Encoding: zstd`) and `ZstdHttpDecoder`, which does not read ClickHouse native framing.
Insertion is unaffected: the client compresses the request body itself and the server auto-detects the codec from the frame header.
## ClickHouse server version
The server available here is **26.8.2.7**, which still frames `compress=1` responses as LZ4 (`0x82`) — verified with `curl -s 'http://localhost:8123/?compress=1' --data-binary 'SELECT 1 FORMAT TSV' | od -An -tx1`, whose 17th byte is `82`. So the end-to-end failure was **not** reproduced against a live 26.9+ server; the reproduction below feeds the decoder exactly the framing a 26.9+ server produces (built with this crate's own ZSTD native-framing writer) and was run.
## Reproduction
Unit test appended to `src/compression/lz4.rs`, run with `cargo test --features lz4,zstd --lib repro_zstd_framed`:
```rust
#[cfg(feature = "zstd")]
#[tokio::test]
async fn repro_zstd_framed_response_rejected() {
use futures_util::stream::{self, TryStreamExt};
let original = b"1\n".to_vec();
// What a ClickHouse 26.9+ server sends for `?compress=1`: native framing
// with the ZSTD method byte (0x90), since the default codec is now ZSTD(3).
let framed = crate::compression::zstd::compress(&original, Some(3)).unwrap();
assert_eq!(framed[16], 0x90);
let stream = stream::iter(vec![Ok::<_, Error>(framed)]);
let mut decoder = Lz4Decoder::new(stream);
match decoder.try_next().await {
Ok(_) => panic!("decoded fine"),
Err(err) => panic!("decoder failed with: {err}"),
}
}
```
Expected: the frame is self-describing, so a native-framing reader should detect method byte `0x90` and decompress it as ZSTD, yielding `1\n`.
Actual:
```
thread 'compression::lz4::repro_zstd_framed_response_rejected' panicked at src/compression/lz4.rs:266:21:
decoder failed with: decompression error: incorrect magic number
```
The equivalent user-facing call on a 26.9+ server (default features, `Compression::Lz4`):
```rust
let client = clickhouse::Client::default().with_url("http://localhost:8123");
// fails with Decompression("incorrect magic number") on 26.9+
let n = client.query("SELECT 1").fetch_one::().await?;
```
## Suggested fix
`Lz4Decoder` (`src/compression/lz4.rs:88-108`, `Lz4Meta::read`) should be generalized into a native-framing decoder that dispatches on the method byte in the frame header rather than asserting `0x82`:
- `0x82` → LZ4 block (current path)
- `0x90` → ZSTD block (`zstd::bulk::decompress` with the header's uncompressed size; the block-level ZSTD write path already exists in `src/compression/zstd.rs`)
- `0x02` → no compression (raw payload after the 9-byte header)
The checksum verification and `total_size` accounting are codec-independent and can stay as they are. Note this also means the `zstd` feature (or a vendored ZSTD decoder) becomes required to read `compress=1` responses from 26.9+ servers, which is worth calling out in the feature docs.
## Link
Same root cause as [ClickHouse/clickhouse-java#3105](https://github.com/ClickHouse/clickhouse-java/issues/3105).
Contributor guide
Research direction
Start with src/compression/lz4.rs, especially Lz4Meta::read and Lz4Decoder, then compare the native-framing block path in src/compression/zstd.rs. Run cargo test --features lz4,zstd --lib repro_zstd_framed; done means native frames dispatch by method byte for LZ4, ZSTD, and uncompressed payloads while preserving checksum and size handling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 65/100