ClickHouse / ClickHouse/clickhouse-java

Insert function is not working

Open
#1,401 9 comments 0 reactions 0 assignees View on GitHub
action:verify area:docs duplicate
Dominant language
Java
Stars
1.6k
Forks
636
Avg merge
2d 23h
Merged PRs (30d)
29

Description

### I am not able to insert the CSV data into the existing table in click house. using clickhouse-java

I run the below code but it throws the errro:
```
static long insert(ClickHouseNode server, String table) throws ClickHouseException {
try (ClickHouseClient client = ClickHouseClient.newInstance(server.getProtocol())) {
ClickHouseRequest.Mutation request = client.connect(server).write().table(table)
.format(ClickHouseFormat.RowBinary);
ClickHouseConfig config = request.getConfig();
CompletableFuture future;
// back-pressuring is not supported, you can adjust the first two arguments
try (ClickHousePipedOutputStream stream = ClickHouseDataStreamFactory.getInstance()
.createPipedOutputStream(config, (Runnable) null)) {
// in async mode, which is default, execution happens in a worker thread
future = request.data(stream.getInputStream()).execute();

// writing happens in main thread
for (int i = 0; i < 10_000; i++) {
BinaryStreamUtils.writeString(stream, String.valueOf(i % 16));
BinaryStreamUtils.writeNonNull(stream);
BinaryStreamUtils.writeString(stream, UUID.randomUUID().toString());
}
}

// response should be always closed
try (ClickHouseResponse response = future.get()) {
ClickHouseResponseSummary summary = response.getSummary();
return summary.getWrittenRows();
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw ClickHouseException.forCancellation(e, server);
} catch (ExecutionException | IOException e) {
throw ClickHouseException.of(e, server);
}
}
```
### Error
```
java.util.concurrent.ExecutionException: com.clickhouse.client.ClickHouseException: Code: 33. DB::Exception: Cannot read all data. Bytes read: 2. Bytes expected: 4.: (at row 14272)
: While executing BinaryRowInputFormat. (CANNOT_READ_ALL_DATA) (version 23.6.2.18 (official build))
, server ClickHouseNode [uri=http://localhost:8123/ShabirBhat_project]@-2130346309
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
at com.datoin.ch.CHclientTest.insert(CHclientTest.java:77)
at com.datoin.ch.CHclientTest.main(CHclientTest.java:40)
Caused by: com.clickhouse.client.ClickHouseException: Code: 33. DB::Exception: Cannot read all data. Bytes read: 2. Bytes expected: 4.: (at row 14272)
: While executing BinaryRowInputFormat. (CANNOT_READ_ALL_DATA) (version 23.6.2.18 (official build))
, server ClickHouseNode [uri=http://localhost:8123/ShabirBhat_project]@-2130346309
at com.clickhouse.client.ClickHouseException.of(ClickHouseException.java:168)
at com.clickhouse.client.AbstractClient.lambda$execute$0(AbstractClient.java:275)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.io.IOException: Code: 33. DB::Exception: Cannot read all data. Bytes read: 2. Bytes expected: 4.: (at row 14272)
: While executing BinaryRowInputFormat. (CANNOT_READ_ALL_DATA) (version 23.6.2.18 (official build))

at com.clickhouse.client.http.HttpUrlConnectionImpl.checkResponse(HttpUrlConnectionImpl.java:184)
at com.clickhouse.client.http.HttpUrlConnectionImpl.post(HttpUrlConnectionImpl.java:227)
at com.clickhouse.client.http.ClickHouseHttpClient.send(ClickHouseHttpClient.java:124)
at com.clickhouse.client.AbstractClient.sendAsync(AbstractClient.java:161)
at com.clickhouse.client.AbstractClient.lambda$execute$0(AbstractClient.java:273)
... 4 more

Process finished with exit code 0

```
# I tried to replace the below code with my csv data

```
// writing happens in main thread
for (int i = 0; i < 10_000; i++) {
BinaryStreamUtils.writeString(stream, String.valueOf(i % 16));
BinaryStreamUtils.writeNonNull(stream);
BinaryStreamUtils.writeString(stream, UUID.randomUUID().toString());
}

Replace with :
BufferedReader reader = new BufferedReader(new FileReader(csvFilePath));
String line;
while ((line = reader.readLine()) != null) {
System.out.println(line);
BinaryStreamUtils.writeString(stream, line);
}
```

Contributor guide

Open the contributing guide

Research direction

Start at the insert method in the reported CHclientTest.java stack trace and review the ClickHouseRequest.Mutation setup using RowBinary. Compare the original BinaryStreamUtils writes with the later CSV line-writing attempt, and check the target table schema and expected input format. Done means the example inserts the CSV data without the CANNOT_READ_ALL_DATA error and reports the written rows.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.