ClickHouse / ClickHouse/clickhouse-java

Clickhouse Closed unexpectedly. Code: 33. DB::Exception: Cannot read all data.

Open
#2,479 2 comments 1 reaction 0 assignees View on GitHub
area:network client-api-v2
Dominant language
Java
Stars
1.6k
Forks
636
Avg merge
2d 16h
Merged PRs (30d)
28

Description

Leaving this as a footprint here in case anyone else runs into this head-scratcher.

I was trying to use the CH Java Client-v2 library to write RowBinary, and was running into this DB Exception error repeatedly even though my data was set up correctly according to the unit tests in https://github.com/ClickHouse/clickhouse-java/blob/main/client-v2/src/test/java/com/clickhouse/client/datatypes/RowBinaryFormatWriterTest.java#L177

**Code:**

client = new Client.Builder()
.addEndpoint(Protocol.HTTP, "####", 8123, false)
.setUsername(builder.user)
.setPassword(builder.pass)
.setDefaultDatabase("public")
.build();

TableSchema schema = client.getTableSchema(table);

ClickHouseFormat format = ClickHouseFormat.RowBinary;
InsertSettings settings = new InsertSettings();
try (InsertResponse response = client.insert(table, out -> {
RowBinaryFormatWriter w = new RowBinaryFormatWriter(out, schema, format);

for(Message message : messages){
Field[] row = transformMessage(message);
for (Field field : row) {
w.setValue(field.index, field.value);
}
w.commitRow();
}
}, format, settings).get()) {
logger.info("Rows written: " + response.getWrittenRows());
}

I can't show it all, but basically it was pretty inline with the unit test

**Error message:**
clickhouse Closed unexpectedly. Code: 33. DB::Exception: Cannot read all data. Bytes read: 1648. Bytes expected: 2097151

Unfortunately this error message was pretty vague, and I can't recreate anymore to get the full dump. The only thing I can notice was odd is that I was trying to insert a single row, and on average that has only 600 bytes, but it expected 2 million... which is way overestimated.

**Solution:**
This error was resolved after i specified the bufferSize in the client configuration.

int bufferSize = (7 * 65500);
client = new Client.Builder()
.addEndpoint(Protocol.HTTP, "####", 8123, false)
.setUsername(builder.user)
.setPassword(builder.pass)
.setDefaultDatabase("public")
.setSocketSndbuf(bufferSize)
.setSocketRcvbuf(bufferSize)
.setClientNetworkBufferSize(bufferSize)
.build();

And remains resolved even after I remove the buffer size config and rerun the application.

So, I don't know why or how this fixed it, but i hope it helps anyone else running into this specific issue

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.