ClickHouse / ClickHouse/clickhouse-java

[client-v2] Every compressed read fails on ClickHouse 26.9+: response reader is hardcoded to LZ4 but the server default codec is now ZSTD(3)

Ouverte
#3,105 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Java
Étoiles
1.6k
Forks
636
Merge moyen
2 j 23 h
PR mergées (30 j)
29

Description

### Summary

ClickHouse [#108786](https://github.com/ClickHouse/ClickHouse/pull/108786) ("Switch the default compression to ZSTD(3) for table data and network", merged 2026-09-05, `Version info` says merged into `26.9.1.810`, so included in 26.9 and later) changes `CompressionCodecFactory::getDefaultCodec` from `LZ4` to `ZSTD(3)`.

The HTTP `compress=1` framed output is one of the direct `getDefaultCodec` users. client-v2 requests `compress=1` by default and pipes the response body through `ClickHouseLZ4InputStream`, which asserts the LZ4 block magic byte, so **every compressed read fails against ClickHouse 26.9+**:

```
com.clickhouse.client.api.ClientException: Invalid LZ4 magic byte: '-112'
at com.clickhouse.client.api.internal.ClickHouseLZ4InputStream.refill(ClickHouseLZ4InputStream.java:112)
at com.clickhouse.client.api.internal.ClickHouseLZ4InputStream.read(ClickHouseLZ4InputStream.java:62)
...
at com.clickhouse.client.api.internal.TableSchemaParser.readTSKV(TableSchemaParser.java:23)
at com.clickhouse.client.api.Client.getTableSchemaImpl(Client.java:1872)
at com.clickhouse.client.api.Client.getTableSchema(Client.java:1847)
```

`getTableSchema` is just where we hit it first — this is not specific to schema introspection, it affects any compressed response.

Per that PR's own changelog, the HTTP `compress=1` path has **no runtime rollback**: it is not controlled by the `compatibility` setting, per-column `CODEC`, or the server `` config. So this cannot be worked around server-side.

### Affected versions

- client-v2 **0.9.5** — verified failing
- client-v2 **0.10.0** (latest release) — same `ClickHouseLZ4InputStream` with the same `Invalid LZ4 magic byte` check, so also affected
- Server: ClickHouse **26.9+** (reproduced on `clickhouse/clickhouse-server:head` = `26.9.1.943`). 26.8 and earlier are unaffected by *this* bug.

### Root cause / evidence

`-112` is `0x90` signed. Raw `curl` against 26.9.1.943 confirms the framing:

```
$ curl -s -u default:x 'localhost:8123/?compress=1' --data-binary 'DESCRIBE TABLE t FORMAT TSKV' | xxd | head -2
00000000: 151e 70e4 4490 8555 b0c4 8424 d0d8 731d ..p.D..U...$..s.
00000010: 9065 0000 00c0 0000 0028 b52f fd20 c09d .e.......(./. ..
```

After the 16-byte checksum, offset `0x10` is `0x90` — the compression method byte for ZSTD, exactly the byte reported in the exception — and offset `0x18` is `28 b5 2f fd`, the ZSTD magic number. `ClickHouseLZ4InputStream` expects `0x82` there.

The same request without `compress=1` returns plain readable TSKV.

### Reproduce

```bash
docker run -d --name chhead -e CLICKHOUSE_PASSWORD=x -p 8123:8123 clickhouse/clickhouse-server:head
# then any client-v2 read, e.g.
# client.getTableSchema("t", "default")
```

### Suggested fix

The compressed frame is self-describing — ClickHouse#108786 explicitly relies on "the receiver auto-detects the codec", which holds for the server's own readers. client-v2 does not auto-detect; it assumes LZ4. The response reader should dispatch on the compression method byte in the frame header (`0x82` LZ4, `0x90` ZSTD, `0x02` none) rather than asserting LZ4, and gain a ZSTD decompression path.

### Note on interaction with #3070 / #3094

On 26.9+ this bug currently *masks* the `X-ClickHouse-Format` precedence bug (#3070, #3094): the stream dies in `refill` before `readTSKV` ever sees bytes. I verified the header bug is still present on 26.9.1.943 — `curl -H 'format: TSKV' ... --data-binary 'DESCRIBE TABLE t'` returns TabSeparated there too. So fixing only this issue will move `getTableSchema` back to failing with `Non-null columnName and columnType are required`; both need to land.

Setting `compressServerResponse(false)` sidesteps this issue alone, but for the same reason it is not a usable workaround for `getTableSchema`, and it costs compression on every read.

### Context

Found while running the [ClickHouse Flink connector](https://github.com/ClickHouse/flink-connector-clickhouse) test suite against ClickHouse HEAD.

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

Start with client-v2's ClickHouseLZ4InputStream.java, especially the refill path at line 112, and trace how compressed HTTP responses reach the response readers used by getTableSchema and other reads. Reproduce against clickhouse/clickhouse-server:head with compress=1, then verify that compressed LZ4, ZSTD, and uncompressed responses are handled according to their frame method bytes without breaking existing reads.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
java
Domaine
databases
Type d'issue
Bug
Difficulté
4/5
Temps estimé
3-5 jours
Activité
Active
Clarté
Plutôt claire
Accessibilité débutants
64/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.