ClickHouse / ClickHouse/clickhouse-java

Clickhouse Closed unexpectedly. Code: 33. DB::Exception: Cannot read all data.

未关闭
#2,479 2 条评论 1 个 reaction 已指派 0 人 在 GitHub 查看
area:network client-api-v2
主要语言
Java
星标
1.6k
派生
636
平均合并
2 天 23 小时
30 天内合并 PR
29

描述

Leaving this as a footprint here in case anyone else runs into this head-scratcher.

I was trying to use the CH Java Client-v2 library to write RowBinary, and was running into this DB Exception error repeatedly even though my data was set up correctly according to the unit tests in https://github.com/ClickHouse/clickhouse-java/blob/main/client-v2/src/test/java/com/clickhouse/client/datatypes/RowBinaryFormatWriterTest.java#L177

**Code:**

client = new Client.Builder()
.addEndpoint(Protocol.HTTP, "####", 8123, false)
.setUsername(builder.user)
.setPassword(builder.pass)
.setDefaultDatabase("public")
.build();

TableSchema schema = client.getTableSchema(table);

ClickHouseFormat format = ClickHouseFormat.RowBinary;
InsertSettings settings = new InsertSettings();
try (InsertResponse response = client.insert(table, out -> {
RowBinaryFormatWriter w = new RowBinaryFormatWriter(out, schema, format);

for(Message message : messages){
Field[] row = transformMessage(message);
for (Field field : row) {
w.setValue(field.index, field.value);
}
w.commitRow();
}
}, format, settings).get()) {
logger.info("Rows written: " + response.getWrittenRows());
}

I can't show it all, but basically it was pretty inline with the unit test

**Error message:**
clickhouse Closed unexpectedly. Code: 33. DB::Exception: Cannot read all data. Bytes read: 1648. Bytes expected: 2097151

Unfortunately this error message was pretty vague, and I can't recreate anymore to get the full dump. The only thing I can notice was odd is that I was trying to insert a single row, and on average that has only 600 bytes, but it expected 2 million... which is way overestimated.

**Solution:**
This error was resolved after i specified the bufferSize in the client configuration.

int bufferSize = (7 * 65500);
client = new Client.Builder()
.addEndpoint(Protocol.HTTP, "####", 8123, false)
.setUsername(builder.user)
.setPassword(builder.pass)
.setDefaultDatabase("public")
.setSocketSndbuf(bufferSize)
.setSocketRcvbuf(bufferSize)
.setClientNetworkBufferSize(bufferSize)
.build();

And remains resolved even after I remove the buffer size config and rerun the application.

So, I don't know why or how this fixed it, but i hope it helps anyone else running into this specific issue

贡献指南

打开贡献指南

调研方向

Start with client-v2/src/test/java/com/clickhouse/client/datatypes/RowBinaryFormatWriterTest.java and the Client.Builder network-buffer configuration shown in the report. Reproduce the RowBinary insert with a single row, compare behavior with and without the buffer settings, and inspect the resulting exception. Done means identifying a reproducible cause and documenting or fixing it; the issue does not specify a code change.

由索引模型根据 Issue 内容生成。

评估

技术栈
clickhouse, java
领域
databases
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
冷清
描述清晰度
需要澄清
新手友好度
25/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。