ClickHouse / ClickHouse/clickhouse-java

Major performance regression in client-v2

オープン
#2,516 コメント 10 件 リアクション 7 件 担当者 0 名 GitHub で見る
client-api-v2 performance v2-feedback
主要言語
Java
スター
1.6k
フォーク
636
平均マージ
2日 23時間
マージ済み PR(30日)
29

説明

### Summary

After migrating from the ClickHouse Java client v1 to v2 we observed a major performance regression: more than 2 times less throughput. We use ClickHouse as a pre-aggregation layer, and run analytical queries that may return 10^6-10^9 rows for later processing. Because our pipeline depends on low-latency, low-overhead reads, even small regressions translate into major throughput losses.

### Reproduction

The benchmark suite in this repository measures the general query performance differences between v1 and v2 but does not cover the most performance-sensitive scenario: retrieving column values using strongly-typed getters (e.g., the reason `java.sql.ResultSet#getLong()` exists). In such cases, the new v2 client exhibits a \~100% performance drop. This regression affects any user who relies on `getXxx()` methods and migrates to the new ClickHouse JDBC driver.

I’ve submitted a [PR]() that introduces 2 new benchmarks using strongly-typed getters. Below are the results from running them on my local machine:

* OpenJDK version: 24.0.2
* macOS 15.5, Apple M3 Pro

```
mvn compile exec:exec -Dexec.executable=java -Dexec.args="-classpath %classpath com.clickhouse.benchmark.BenchmarkRunner -m 3 -b q -l 300000"

QueryClient.queryV1 110 276.252 ± 31.343 ms/op
QueryClient.queryV2 125 245.277 ± 20.057 ms/op
QueryClient.queryV1WithTypes 144 209.260 ± 17.059 ms/op
QueryClient.queryV2WithTypes 68 454.188 ± 36.329 ms/op
```

As you can see, `QueryClient.queryV1` and `QueryClient.queryV2` perform similarly. However, `QueryClient.queryV2WithTypes` is more than 2x slower than `QueryClient.queryV1WithTypes`.
While the gap between `QueryClient.queryV2` and `QueryClient.queryV1WithTypes` is around 20%, memory allocations are significantly higher in v2, increasing GC pressure which usually run concurrently.

### Root cause

Two main differences in v2 contribute to the regression, both of which are not present in v1:

* It creates a new `Object[]` for every row. This means every read triggers an array allocation, and all primitive values are boxed. This increases GC pressure and negatively impacts data locality.
In contrast, v1 reuses a single array with mutable wrappers to store values, avoiding these allocations entirely (see `ClickHouseClientOption#REUSE_VALUE_WRAPPER`).
* In v2, reading a primitive value like `getLong(int)` involves a chain of unnecessary hash table lookups:
`com.clickhouse.client.api.metadata.TableSchema#columnIndexToName` -> `nameToIndex` -> `nameToIndex`. `com.clickhouse.jdbc.ResultSetImpl#getLong(int)` adds one more lookup on top of that chain. This results in 4 `HashMap.get()` calls per column access by index to read every primitive column value.

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

PR #2515で導入されたベンチマークから始め、issueに記載されたMaven BenchmarkRunnerコマンドを実行して、型付きgetterの差異を再現します。TableSchema#columnIndexToNameとResultSetImpl#getLong(int)、およびv1 ClickHouseClientOption#REUSE_VALUE_WRAPPERの動作を読みます。型付きgetterで報告されたリグレッションが発生しなくなり、ベンチマーク結果が文書化されていれば完了です。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
java
領域
databases, performance
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
停滞
明瞭さ
おおむね明確
初心者へのやさしさ
28/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。