duckdb / duckdb/duckdb-java

Connection#createArrayOf performance worse than passing in array literals

Open
#68 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
127
Forks
80
Avg merge
13h 49m
Merged PRs (30d)
48

Description

I originally commented on the MR that implements this.

I was excited to see support for `Connection#createArrayOf` was added so I decided to build locally to try it out and I came across one issue and a question on performance.

Before I saw that this had been implemented I passed arrays as literals from java then casting the parameter to the array type in the sql, it seems passing the array literal is many times faster than using `Connection#createArrayOf`, I tried similar on postgres in the past and the speed improvement using `Connection#createArrayOf` was significant.
Quick test code (using spring)
```java
public static void main(String[] args) throws SQLException {
var jdbcClient = JdbcClient.create(new SingleConnectionDataSource("jdbc:duckdb:", true));

jdbcClient.sql("create table foo(i int)")
.update();

String arrayLiteral = IntStream.range(0, 10000).mapToObj(String::valueOf).collect(Collectors.joining(",", "[", "]"));

System.out.println("ArrayLiteral");

for (int i = 0; i < 100; i++) {
final var now2 = Instant.now();
jdbcClient.sql("insert into foo select unnest(:arr::int[])")
.param("arr", arrayLiteral)
.update();
final var duration = Duration.between(now2, Instant.now());
System.out.println(duration.toNanos() / 1000000d);
}

jdbcClient.sql("truncate table foo")
.update();

System.out.println("`Connection#createArrayOf`");

final var elements = IntStream.range(0, 10000).boxed().toArray();

try (final var connection = new SingleConnectionDataSource("jdbc:duckdb:", true).getConnection()) {
for (int i = 0; i < 100; i++) {
final var now = Instant.now();
jdbcClient.sql("insert into foo select unnest(:arr::int[])")
.param("arr", connection.createArrayOf("int4", elements))
.update();
final var duration = Duration.between(now, Instant.now());
System.out.println(duration.toNanos() / 1000000d);
}
}
}
```
output on M1 mac showing 1.3ms for array literal vs 6.4ms for `Connection#createArrayOf`, is this expected as a first implementation or do you think something is a rye?

unfortunately, I've not got the time to dig into potential performance improvements on the java side and I don't know c++ so can't help there 😓

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.