ClickHouse / ClickHouse/clickhouse-java

Insert function is not working

未关闭
#1,401 9 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
action:verify area:docs duplicate
主要语言
Java
星标
1.6k
派生
636
平均合并
2 天 23 小时
30 天内合并 PR
29

描述

### I am not able to insert the CSV data into the existing table in click house. using clickhouse-java

I run the below code but it throws the errro:
```
static long insert(ClickHouseNode server, String table) throws ClickHouseException {
try (ClickHouseClient client = ClickHouseClient.newInstance(server.getProtocol())) {
ClickHouseRequest.Mutation request = client.connect(server).write().table(table)
.format(ClickHouseFormat.RowBinary);
ClickHouseConfig config = request.getConfig();
CompletableFuture future;
// back-pressuring is not supported, you can adjust the first two arguments
try (ClickHousePipedOutputStream stream = ClickHouseDataStreamFactory.getInstance()
.createPipedOutputStream(config, (Runnable) null)) {
// in async mode, which is default, execution happens in a worker thread
future = request.data(stream.getInputStream()).execute();

// writing happens in main thread
for (int i = 0; i < 10_000; i++) {
BinaryStreamUtils.writeString(stream, String.valueOf(i % 16));
BinaryStreamUtils.writeNonNull(stream);
BinaryStreamUtils.writeString(stream, UUID.randomUUID().toString());
}
}

// response should be always closed
try (ClickHouseResponse response = future.get()) {
ClickHouseResponseSummary summary = response.getSummary();
return summary.getWrittenRows();
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw ClickHouseException.forCancellation(e, server);
} catch (ExecutionException | IOException e) {
throw ClickHouseException.of(e, server);
}
}
```
### Error
```
java.util.concurrent.ExecutionException: com.clickhouse.client.ClickHouseException: Code: 33. DB::Exception: Cannot read all data. Bytes read: 2. Bytes expected: 4.: (at row 14272)
: While executing BinaryRowInputFormat. (CANNOT_READ_ALL_DATA) (version 23.6.2.18 (official build))
, server ClickHouseNode [uri=http://localhost:8123/ShabirBhat_project]@-2130346309
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
at com.datoin.ch.CHclientTest.insert(CHclientTest.java:77)
at com.datoin.ch.CHclientTest.main(CHclientTest.java:40)
Caused by: com.clickhouse.client.ClickHouseException: Code: 33. DB::Exception: Cannot read all data. Bytes read: 2. Bytes expected: 4.: (at row 14272)
: While executing BinaryRowInputFormat. (CANNOT_READ_ALL_DATA) (version 23.6.2.18 (official build))
, server ClickHouseNode [uri=http://localhost:8123/ShabirBhat_project]@-2130346309
at com.clickhouse.client.ClickHouseException.of(ClickHouseException.java:168)
at com.clickhouse.client.AbstractClient.lambda$execute$0(AbstractClient.java:275)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.io.IOException: Code: 33. DB::Exception: Cannot read all data. Bytes read: 2. Bytes expected: 4.: (at row 14272)
: While executing BinaryRowInputFormat. (CANNOT_READ_ALL_DATA) (version 23.6.2.18 (official build))

at com.clickhouse.client.http.HttpUrlConnectionImpl.checkResponse(HttpUrlConnectionImpl.java:184)
at com.clickhouse.client.http.HttpUrlConnectionImpl.post(HttpUrlConnectionImpl.java:227)
at com.clickhouse.client.http.ClickHouseHttpClient.send(ClickHouseHttpClient.java:124)
at com.clickhouse.client.AbstractClient.sendAsync(AbstractClient.java:161)
at com.clickhouse.client.AbstractClient.lambda$execute$0(AbstractClient.java:273)
... 4 more

Process finished with exit code 0

```
# I tried to replace the below code with my csv data

```
// writing happens in main thread
for (int i = 0; i < 10_000; i++) {
BinaryStreamUtils.writeString(stream, String.valueOf(i % 16));
BinaryStreamUtils.writeNonNull(stream);
BinaryStreamUtils.writeString(stream, UUID.randomUUID().toString());
}

Replace with :
BufferedReader reader = new BufferedReader(new FileReader(csvFilePath));
String line;
while ((line = reader.readLine()) != null) {
System.out.println(line);
BinaryStreamUtils.writeString(stream, line);
}
```

贡献指南

打开贡献指南

调研方向

Start at the insert method in the reported CHclientTest.java stack trace and review the ClickHouseRequest.Mutation setup using RowBinary. Compare the original BinaryStreamUtils writes with the later CSV line-writing attempt, and check the target table schema and expected input format. Done means the example inserts the CSV data without the CANNOT_READ_ALL_DATA error and reports the written rows.

由索引模型根据 Issue 内容生成。

评估

技术栈
java
领域
databases
Issue 类型
缺陷
难度
3/5
预计耗时
1-2 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。