ClickHouse / ClickHouse/clickhouse-java

Slow deserialization of DateTime values

Open
#715 2 comments 0 reactions 0 assignees View on GitHub
area:data-type performance test
Dominant language
Java
Stars
1.6k
Forks
636
Avg merge
2d 23h
Merged PRs (30d)
29

Description

Deserialization of DateTime value seems very slow comparing to string and number.

Below are benchmarks against ClickHouse 21.8.
```
Benchmark (client) (connection) (statement) Mode Cnt Score Error Units
Query.select10kStringRows clickhouse-jdbc(0.3.1-patch) reuse normal thrpt 20 55.892 ± 3.277 ops/s
Query.select10kTimestampRows clickhouse-jdbc(0.3.1-patch) reuse normal thrpt 20 10.243 ± 0.627 ops/s
Query.select10kUInt64Rows clickhouse-jdbc(0.3.1-patch) reuse normal thrpt 20 54.420 ± 4.530 ops/s
Query.select10kStringRows clickhouse-native-jdbc-shaded(2.5.6) reuse normal thrpt 20 73.846 ± 5.486 ops/s
Query.select10kTimestampRows clickhouse-native-jdbc-shaded(2.5.6) reuse normal thrpt 20 83.446 ± 5.502 ops/s
Query.select10kUInt64Rows clickhouse-native-jdbc-shaded(2.5.6) reuse normal thrpt 20 83.512 ± 4.226 ops/s
```

`clickhouse-grpc-client`(0.3.2) with grpc compression disabled on server:
```
Benchmark (connection) (format) (mode) (protocol) (transport) Mode Cnt Score Error Units
Query.select10kStringRows reuse RowBinaryWithNamesAndTypes sync GRPC netty thrpt 20 82.786 ± 4.334 ops/s
Query.select10kStringRows reuse RowBinaryWithNamesAndTypes sync GRPC okhttp thrpt 20 70.753 ± 4.797 ops/s
Query.select10kStringRows reuse TabSeparatedWithNamesAndTypes sync GRPC netty thrpt 20 60.482 ± 4.466 ops/s
Query.select10kStringRows reuse TabSeparatedWithNamesAndTypes sync GRPC okhttp thrpt 20 52.046 ± 2.540 ops/s
Query.select10kTimestampRows reuse RowBinaryWithNamesAndTypes sync GRPC netty thrpt 20 99.147 ± 5.190 ops/s
Query.select10kTimestampRows reuse RowBinaryWithNamesAndTypes sync GRPC okhttp thrpt 20 86.454 ± 5.485 ops/s
Query.select10kTimestampRows reuse TabSeparatedWithNamesAndTypes sync GRPC netty thrpt 20 48.739 ± 3.207 ops/s
Query.select10kTimestampRows reuse TabSeparatedWithNamesAndTypes sync GRPC okhttp thrpt 20 29.759 ± 1.812 ops/s
Query.select10kUInt64Rows reuse RowBinaryWithNamesAndTypes sync GRPC netty thrpt 20 121.422 ± 6.465 ops/s
Query.select10kUInt64Rows reuse RowBinaryWithNamesAndTypes sync GRPC okhttp thrpt 20 73.226 ± 3.930 ops/s
Query.select10kUInt64Rows reuse TabSeparatedWithNamesAndTypes sync GRPC netty thrpt 20 66.267 ± 4.822 ops/s
Query.select10kUInt64Rows reuse TabSeparatedWithNamesAndTypes sync GRPC okhttp thrpt 20 56.022 ± 4.188 ops/s
```

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the Query.select10kTimestampRows benchmark and compare its DateTime results with the string and UInt64 cases across RowBinaryWithNamesAndTypes and TabSeparatedWithNamesAndTypes. Review the deserialization path used by the Java clients and the related work represented by pull request #736; done means identifying and addressing the DateTime throughput gap with benchmark evidence.

Written by the indexing model from the issue text.

Assessment

Tech stack
grpc, java
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.