ClickHouse / ClickHouse/clickhouse-java

Clickhouse Closed unexpectedly. Code: 33. DB::Exception: Cannot read all data.

オープン
#2,479 コメント 2 件 リアクション 1 件 担当者 0 名 GitHub で見る
area:network client-api-v2
主要言語
Java
スター
1.6k
フォーク
636
平均マージ
2日 23時間
マージ済み PR(30日)
29

説明

Leaving this as a footprint here in case anyone else runs into this head-scratcher.

I was trying to use the CH Java Client-v2 library to write RowBinary, and was running into this DB Exception error repeatedly even though my data was set up correctly according to the unit tests in https://github.com/ClickHouse/clickhouse-java/blob/main/client-v2/src/test/java/com/clickhouse/client/datatypes/RowBinaryFormatWriterTest.java#L177

**Code:**

client = new Client.Builder()
.addEndpoint(Protocol.HTTP, "####", 8123, false)
.setUsername(builder.user)
.setPassword(builder.pass)
.setDefaultDatabase("public")
.build();

TableSchema schema = client.getTableSchema(table);

ClickHouseFormat format = ClickHouseFormat.RowBinary;
InsertSettings settings = new InsertSettings();
try (InsertResponse response = client.insert(table, out -> {
RowBinaryFormatWriter w = new RowBinaryFormatWriter(out, schema, format);

for(Message message : messages){
Field[] row = transformMessage(message);
for (Field field : row) {
w.setValue(field.index, field.value);
}
w.commitRow();
}
}, format, settings).get()) {
logger.info("Rows written: " + response.getWrittenRows());
}

I can't show it all, but basically it was pretty inline with the unit test

**Error message:**
clickhouse Closed unexpectedly. Code: 33. DB::Exception: Cannot read all data. Bytes read: 1648. Bytes expected: 2097151

Unfortunately this error message was pretty vague, and I can't recreate anymore to get the full dump. The only thing I can notice was odd is that I was trying to insert a single row, and on average that has only 600 bytes, but it expected 2 million... which is way overestimated.

**Solution:**
This error was resolved after i specified the bufferSize in the client configuration.

int bufferSize = (7 * 65500);
client = new Client.Builder()
.addEndpoint(Protocol.HTTP, "####", 8123, false)
.setUsername(builder.user)
.setPassword(builder.pass)
.setDefaultDatabase("public")
.setSocketSndbuf(bufferSize)
.setSocketRcvbuf(bufferSize)
.setClientNetworkBufferSize(bufferSize)
.build();

And remains resolved even after I remove the buffer size config and rerun the application.

So, I don't know why or how this fixed it, but i hope it helps anyone else running into this specific issue

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

Start with client-v2/src/test/java/com/clickhouse/client/datatypes/RowBinaryFormatWriterTest.java and the Client.Builder network-buffer configuration shown in the report. Reproduce the RowBinary insert with a single row, compare behavior with and without the buffer settings, and inspect the resulting exception. Done means identifying a reproducible cause and documenting or fixing it; the issue does not specify a code change.

索引モデルが issue の本文から書いたものです。

評価

技術スタック
clickhouse, java
領域
databases
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
静か
明瞭さ
説明が足りない
初心者へのやさしさ
25/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。