ClickHouse / ClickHouse/clickhouse-java

Clickhouse Closed unexpectedly. Code: 33. DB::Exception: Cannot read all data.

Aperta
#2,479 2 commenti 1 reazione 0 assegnatari Vedi su GitHub
area:network client-api-v2
Lingua principale
Java
Stelle
1.6k
Fork
636
Merge medio
2g 23h
PR unite (30g)
29

Descrizione

Leaving this as a footprint here in case anyone else runs into this head-scratcher.

I was trying to use the CH Java Client-v2 library to write RowBinary, and was running into this DB Exception error repeatedly even though my data was set up correctly according to the unit tests in https://github.com/ClickHouse/clickhouse-java/blob/main/client-v2/src/test/java/com/clickhouse/client/datatypes/RowBinaryFormatWriterTest.java#L177

**Code:**

client = new Client.Builder()
.addEndpoint(Protocol.HTTP, "####", 8123, false)
.setUsername(builder.user)
.setPassword(builder.pass)
.setDefaultDatabase("public")
.build();

TableSchema schema = client.getTableSchema(table);

ClickHouseFormat format = ClickHouseFormat.RowBinary;
InsertSettings settings = new InsertSettings();
try (InsertResponse response = client.insert(table, out -> {
RowBinaryFormatWriter w = new RowBinaryFormatWriter(out, schema, format);

for(Message message : messages){
Field[] row = transformMessage(message);
for (Field field : row) {
w.setValue(field.index, field.value);
}
w.commitRow();
}
}, format, settings).get()) {
logger.info("Rows written: " + response.getWrittenRows());
}

I can't show it all, but basically it was pretty inline with the unit test

**Error message:**
clickhouse Closed unexpectedly. Code: 33. DB::Exception: Cannot read all data. Bytes read: 1648. Bytes expected: 2097151

Unfortunately this error message was pretty vague, and I can't recreate anymore to get the full dump. The only thing I can notice was odd is that I was trying to insert a single row, and on average that has only 600 bytes, but it expected 2 million... which is way overestimated.

**Solution:**
This error was resolved after i specified the bufferSize in the client configuration.

int bufferSize = (7 * 65500);
client = new Client.Builder()
.addEndpoint(Protocol.HTTP, "####", 8123, false)
.setUsername(builder.user)
.setPassword(builder.pass)
.setDefaultDatabase("public")
.setSocketSndbuf(bufferSize)
.setSocketRcvbuf(bufferSize)
.setClientNetworkBufferSize(bufferSize)
.build();

And remains resolved even after I remove the buffer size config and rerun the application.

So, I don't know why or how this fixed it, but i hope it helps anyone else running into this specific issue

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.