ClickHouse / ClickHouse/clickhouse-java

Clickhouse Closed unexpectedly. Code: 33. DB::Exception: Cannot read all data.

Offen
#2,479 2 Kommentare 1 Reaktion 0 zugewiesene Personen Auf GitHub ansehen
area:network client-api-v2
Vorherrschende Sprache
Java
Sterne
1.6k
Forks
636
Ø Merge
2 T. 23 Std.
Gemergte PRs (30 T.)
29

Beschreibung

Leaving this as a footprint here in case anyone else runs into this head-scratcher.

I was trying to use the CH Java Client-v2 library to write RowBinary, and was running into this DB Exception error repeatedly even though my data was set up correctly according to the unit tests in https://github.com/ClickHouse/clickhouse-java/blob/main/client-v2/src/test/java/com/clickhouse/client/datatypes/RowBinaryFormatWriterTest.java#L177

**Code:**

client = new Client.Builder()
.addEndpoint(Protocol.HTTP, "####", 8123, false)
.setUsername(builder.user)
.setPassword(builder.pass)
.setDefaultDatabase("public")
.build();

TableSchema schema = client.getTableSchema(table);

ClickHouseFormat format = ClickHouseFormat.RowBinary;
InsertSettings settings = new InsertSettings();
try (InsertResponse response = client.insert(table, out -> {
RowBinaryFormatWriter w = new RowBinaryFormatWriter(out, schema, format);

for(Message message : messages){
Field[] row = transformMessage(message);
for (Field field : row) {
w.setValue(field.index, field.value);
}
w.commitRow();
}
}, format, settings).get()) {
logger.info("Rows written: " + response.getWrittenRows());
}

I can't show it all, but basically it was pretty inline with the unit test

**Error message:**
clickhouse Closed unexpectedly. Code: 33. DB::Exception: Cannot read all data. Bytes read: 1648. Bytes expected: 2097151

Unfortunately this error message was pretty vague, and I can't recreate anymore to get the full dump. The only thing I can notice was odd is that I was trying to insert a single row, and on average that has only 600 bytes, but it expected 2 million... which is way overestimated.

**Solution:**
This error was resolved after i specified the bufferSize in the client configuration.

int bufferSize = (7 * 65500);
client = new Client.Builder()
.addEndpoint(Protocol.HTTP, "####", 8123, false)
.setUsername(builder.user)
.setPassword(builder.pass)
.setDefaultDatabase("public")
.setSocketSndbuf(bufferSize)
.setSocketRcvbuf(bufferSize)
.setClientNetworkBufferSize(bufferSize)
.build();

And remains resolved even after I remove the buffer size config and rerun the application.

So, I don't know why or how this fixed it, but i hope it helps anyone else running into this specific issue

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.