ClickHouse / ClickHouse/clickhouse-java

Batch Insert r2dbc

Open
#1,802 5 comments 0 reactions 0 assignees View on GitHub
bug r2dbc
Dominant language
Java
Stars
1.6k
Forks
636
Avg merge
2d 23h
Merged PRs (30d)
29

Description

Looks like bulk inserts are inserting 1 by 1 and not in batches.

I have created the following table:

```
CREATE TABLE IF NOT EXISTS test
(
`a` Int16,
`b` Int16,
`created` DateTime
)
ENGINE = MergeTree()
PARTITION BY (toYYYYMM(created))
ORDER BY (created)
TTL created + INTERVAL 13 MONTH;
```

If I try to do the following:

```
public Mono saveAll(){
return Mono.from(connectionFactory.create())
.flatMapMany(conn -> execute(conn)).then();
}

private Publisher execute(Connection conn) {
return conn.createStatement("insert into test values (:a, :b, :created)")
.bind("a",1)
.bind("b", 2)
.bind("created", LocalDateTime.now())
.add()
.bind("a",5)
.bind("b", 6)
.bind("created", LocalDateTime.now())
.execute();
}
```
When checking the logs, I can see that 2 connections are established and inserts are run independently so it is not acting as a batch insert.

```
2024-09-02T10:43:28.253Z DEBUG 49005 --- [ckHouseWorker-1] com.clickhouse.client.AbstractClient : Connection established: com.clickhouse.client.http.HttpUrlConnectionImpl@1f010075
2024-09-02T10:43:28.253Z DEBUG 49005 --- [ckHouseWorker-1] c.c.client.http.ClickHouseHttpClient : Query: insert into `default`.test values (1, 2, 1725273808)
2024-09-02T10:43:28.254Z DEBUG 49005 --- [ckHouseWorker-2] com.clickhouse.client.AbstractClient : Connection established: com.clickhouse.client.http.HttpUrlConnectionImpl@35e8f1b6
2024-09-02T10:43:28.254Z DEBUG 49005 --- [ckHouseWorker-2] c.c.client.http.ClickHouseHttpClient : Query: insert into `default`.test values (5, 6, 1725273808)
```

Am I doing something wrong or is this the intended behaviour?

I have also tried using this approach but same behaviour

```
return conn.createBatch()
.add("insert into test values (1, 2, '2024-01-17 00:00:00')")
.add("insert into test values (5, 6, '2024-01-17 00:00:00')")
.execute();
```

I am using clickhouse-r2dbc version 0.6.4.

Contributor guide

Open the contributing guide

Research direction

Reproduce the example against clickhouse-r2dbc 0.6.4, beginning with the Connection.createStatement(...).add() and Connection.createBatch() entry points shown in the issue. Compare their behavior with the logged independent inserts and determine whether the observed semantics are intended. Done means the batch behavior is established and the issue is resolved through the appropriate driver change or documented behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.