ClickHouse / ClickHouse/clickhouse-java
Insert function is not working
- 主要语言
- Java
- 星标
- 1.6k
- 派生
- 636
- 平均合并
- 2 天 23 小时
- 30 天内合并 PR
- 29
描述
### I am not able to insert the CSV data into the existing table in click house. using clickhouse-java
I run the below code but it throws the errro:
```
static long insert(ClickHouseNode server, String table) throws ClickHouseException {
try (ClickHouseClient client = ClickHouseClient.newInstance(server.getProtocol())) {
ClickHouseRequest.Mutation request = client.connect(server).write().table(table)
.format(ClickHouseFormat.RowBinary);
ClickHouseConfig config = request.getConfig();
CompletableFuture future;
// back-pressuring is not supported, you can adjust the first two arguments
try (ClickHousePipedOutputStream stream = ClickHouseDataStreamFactory.getInstance()
.createPipedOutputStream(config, (Runnable) null)) {
// in async mode, which is default, execution happens in a worker thread
future = request.data(stream.getInputStream()).execute();
// writing happens in main thread
for (int i = 0; i < 10_000; i++) {
BinaryStreamUtils.writeString(stream, String.valueOf(i % 16));
BinaryStreamUtils.writeNonNull(stream);
BinaryStreamUtils.writeString(stream, UUID.randomUUID().toString());
}
}
// response should be always closed
try (ClickHouseResponse response = future.get()) {
ClickHouseResponseSummary summary = response.getSummary();
return summary.getWrittenRows();
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw ClickHouseException.forCancellation(e, server);
} catch (ExecutionException | IOException e) {
throw ClickHouseException.of(e, server);
}
}
```
### Error
```
java.util.concurrent.ExecutionException: com.clickhouse.client.ClickHouseException: Code: 33. DB::Exception: Cannot read all data. Bytes read: 2. Bytes expected: 4.: (at row 14272)
: While executing BinaryRowInputFormat. (CANNOT_READ_ALL_DATA) (version 23.6.2.18 (official build))
, server ClickHouseNode [uri=http://localhost:8123/ShabirBhat_project]@-2130346309
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
at com.datoin.ch.CHclientTest.insert(CHclientTest.java:77)
at com.datoin.ch.CHclientTest.main(CHclientTest.java:40)
Caused by: com.clickhouse.client.ClickHouseException: Code: 33. DB::Exception: Cannot read all data. Bytes read: 2. Bytes expected: 4.: (at row 14272)
: While executing BinaryRowInputFormat. (CANNOT_READ_ALL_DATA) (version 23.6.2.18 (official build))
, server ClickHouseNode [uri=http://localhost:8123/ShabirBhat_project]@-2130346309
at com.clickhouse.client.ClickHouseException.of(ClickHouseException.java:168)
at com.clickhouse.client.AbstractClient.lambda$execute$0(AbstractClient.java:275)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.io.IOException: Code: 33. DB::Exception: Cannot read all data. Bytes read: 2. Bytes expected: 4.: (at row 14272)
: While executing BinaryRowInputFormat. (CANNOT_READ_ALL_DATA) (version 23.6.2.18 (official build))
at com.clickhouse.client.http.HttpUrlConnectionImpl.checkResponse(HttpUrlConnectionImpl.java:184)
at com.clickhouse.client.http.HttpUrlConnectionImpl.post(HttpUrlConnectionImpl.java:227)
at com.clickhouse.client.http.ClickHouseHttpClient.send(ClickHouseHttpClient.java:124)
at com.clickhouse.client.AbstractClient.sendAsync(AbstractClient.java:161)
at com.clickhouse.client.AbstractClient.lambda$execute$0(AbstractClient.java:273)
... 4 more
Process finished with exit code 0
```
# I tried to replace the below code with my csv data
```
// writing happens in main thread
for (int i = 0; i < 10_000; i++) {
BinaryStreamUtils.writeString(stream, String.valueOf(i % 16));
BinaryStreamUtils.writeNonNull(stream);
BinaryStreamUtils.writeString(stream, UUID.randomUUID().toString());
}
Replace with :
BufferedReader reader = new BufferedReader(new FileReader(csvFilePath));
String line;
while ((line = reader.readLine()) != null) {
System.out.println(line);
BinaryStreamUtils.writeString(stream, line);
}
```
贡献指南
调研方向
Start at the insert method in the reported CHclientTest.java stack trace and review the ClickHouseRequest.Mutation setup using RowBinary. Compare the original BinaryStreamUtils writes with the later CSV line-writing attempt, and check the target table schema and expected input format. Done means the example inserts the CSV data without the CANNOT_READ_ALL_DATA error and reports the written rows.
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- java
- 领域
- databases
- Issue 类型
- 缺陷
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100