ClickHouse / ClickHouse/clickhouse-rs

Default (LZ4) compression breaks on ClickHouse 26.9+: Lz4Decoder hardcodes magic 0x82 but the server default codec is now ZSTD(3)

Open
#468 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
559
Forks
172
Avg merge
1d 19h
Merged PRs (30d)
3

Description

## Description

`clickhouse-rs` with its default feature set uses `Compression::Lz4`, which appends `?compress=1` to every read request (`src/query.rs:258-267`) and pipes the response body through `Lz4Decoder` (`src/response.rs:284-292`). `Lz4Decoder` dispatches on nothing — it asserts the LZ4 native-framing method byte:

```rust
// src/compression/lz4.rs
const LZ4_MAGIC: u8 = 0x82;
...
if magic != LZ4_MAGIC {
return Err(Error::Decompression("incorrect magic number".into()));
}
```

ClickHouse [#108786](https://github.com/ClickHouse/ClickHouse/pull/108786) ("Switch the default compression to ZSTD(3) for table data and network", in `26.9.1.810` and later) changes `CompressionCodecFactory::getDefaultCodec` from `LZ4` to `ZSTD(3)`. The HTTP `compress=1` framed output is one of the direct `getDefaultCodec` users, so on 26.9+ the server frames the response with method byte `0x90` (ZSTD) instead of `0x82`.

Consequence: with the default (`lz4`) feature set, **every read fails** on ClickHouse 26.9+ with:

```
Error: Decompression("incorrect magic number")
```

Per the upstream PR's own changelog the HTTP `compress=1` path has no runtime rollback (not controlled by `compatibility`, per-column `CODEC`, or the server `` config), so this can't be worked around server-side.

Workarounds on the client side:
- `Client::with_compression(Compression::None)` — loses compression entirely.
- `Client::with_compression(Compression::Zstd(_))` with the `zstd` feature — this path is **not** affected, because it uses HTTP-level compression (`enable_http_compression=1` + `Accept-Encoding: zstd`) and `ZstdHttpDecoder`, which does not read ClickHouse native framing.

Insertion is unaffected: the client compresses the request body itself and the server auto-detects the codec from the frame header.

## ClickHouse server version

The server available here is **26.8.2.7**, which still frames `compress=1` responses as LZ4 (`0x82`) — verified with `curl -s 'http://localhost:8123/?compress=1' --data-binary 'SELECT 1 FORMAT TSV' | od -An -tx1`, whose 17th byte is `82`. So the end-to-end failure was **not** reproduced against a live 26.9+ server; the reproduction below feeds the decoder exactly the framing a 26.9+ server produces (built with this crate's own ZSTD native-framing writer) and was run.

## Reproduction

Unit test appended to `src/compression/lz4.rs`, run with `cargo test --features lz4,zstd --lib repro_zstd_framed`:

```rust
#[cfg(feature = "zstd")]
#[tokio::test]
async fn repro_zstd_framed_response_rejected() {
use futures_util::stream::{self, TryStreamExt};

let original = b"1\n".to_vec();
// What a ClickHouse 26.9+ server sends for `?compress=1`: native framing
// with the ZSTD method byte (0x90), since the default codec is now ZSTD(3).
let framed = crate::compression::zstd::compress(&original, Some(3)).unwrap();
assert_eq!(framed[16], 0x90);

let stream = stream::iter(vec![Ok::<_, Error>(framed)]);
let mut decoder = Lz4Decoder::new(stream);
match decoder.try_next().await {
Ok(_) => panic!("decoded fine"),
Err(err) => panic!("decoder failed with: {err}"),
}
}
```

Expected: the frame is self-describing, so a native-framing reader should detect method byte `0x90` and decompress it as ZSTD, yielding `1\n`.

Actual:

```
thread 'compression::lz4::repro_zstd_framed_response_rejected' panicked at src/compression/lz4.rs:266:21:
decoder failed with: decompression error: incorrect magic number
```

The equivalent user-facing call on a 26.9+ server (default features, `Compression::Lz4`):

```rust
let client = clickhouse::Client::default().with_url("http://localhost:8123");
// fails with Decompression("incorrect magic number") on 26.9+
let n = client.query("SELECT 1").fetch_one::().await?;
```

## Suggested fix

`Lz4Decoder` (`src/compression/lz4.rs:88-108`, `Lz4Meta::read`) should be generalized into a native-framing decoder that dispatches on the method byte in the frame header rather than asserting `0x82`:

- `0x82` → LZ4 block (current path)
- `0x90` → ZSTD block (`zstd::bulk::decompress` with the header's uncompressed size; the block-level ZSTD write path already exists in `src/compression/zstd.rs`)
- `0x02` → no compression (raw payload after the 9-byte header)

The checksum verification and `total_size` accounting are codec-independent and can stay as they are. Note this also means the `zstd` feature (or a vendored ZSTD decoder) becomes required to read `compress=1` responses from 26.9+ servers, which is worth calling out in the feature docs.

## Link

Same root cause as [ClickHouse/clickhouse-java#3105](https://github.com/ClickHouse/clickhouse-java/issues/3105).

Contributor guide

Open the contributing guide

Research direction

Start with src/compression/lz4.rs, especially Lz4Meta::read and Lz4Decoder, then compare the native-framing block path in src/compression/zstd.rs. Run cargo test --features lz4,zstd --lib repro_zstd_framed; done means native frames dispatch by method byte for LZ4, ZSTD, and uncompressed payloads while preserving checksum and size handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.