duckdb / duckdb/duckdb-java

Error when appending to DuckLake table

Open
#249 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
127
Forks
80
Avg merge
13h 49m
Merged PRs (30d)
48

Description

When I want to append many records to a DuckLake table with an appender, I get the following error when closing the `DuckDBAppender` instance. I used `duckdb-java` version 1.3.0.0.
```
java.sql.SQLException: INTERNAL Error: Calling GetStorage on a TableCatalogEntry that is not a DuckTableEntry
This error signals an assertion failure within DuckDB. This usually occurs due to unexpected conditions or errors in the program's logic.
For more information, see https://duckdb.org/docs/stable/dev/internal_errors
at org.duckdb.DuckDBNative.duckdb_jdbc_appender_close(Native Method)
at org.duckdb.DuckDBAppender.close(DuckDBAppender.java:102)
at test.DuckLakeAppenderTest.fails_appender_ducklake(DuckLakeAppenderTest.java:32)
at java.base/java.lang.reflect.Method.invoke(Method.java:565)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1604)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1604)
```

The following test cases may help to reproduce the error.
```java
package test;

import static org.assertj.core.api.BDDAssertions.*;

import java.sql.DriverManager;
import java.sql.SQLException;

import org.duckdb.DuckDBConnection;
import org.junit.jupiter.api.Test;

class DuckLakeAppenderTest {

@Test
void fails_appender_ducklake() throws SQLException {
try (
var connection = (DuckDBConnection) DriverManager.getConnection("jdbc:duckdb:");
var statement = connection.createStatement()
) {
// not necessary with duckdb 1.3.1
connection.setAutoCommit(false);

statement.execute("""
ATTACH 'ducklake:/tmp/test_fails_appender_ducklake.ducklake' AS my_ducklake;
USE my_ducklake;
CREATE TABLE IF NOT EXISTS test (a INT);
""");

try (var appender = connection.createAppender("main", "test")) {
appender.beginRow();
appender.append(124);
appender.endRow();
// fails: INTERNAL Error: Calling GetStorage on a TableCatalogEntry that is not a DuckTableEntry
}

try (var rs = statement.executeQuery("select * from test")) {
rs.next();
then(rs.getInt(1)).isEqualTo(124);
}
}
}

// workaround via temporary table
@Test
void succeeds_appender_ducklake_via_temporary_table() throws SQLException {
try (
var connection = (DuckDBConnection) DriverManager.getConnection("jdbc:duckdb:");
var statement = connection.createStatement()
) {
connection.setAutoCommit(false);

statement.execute("""
CREATE TEMP TABLE IF NOT EXISTS test_temp (a INT);
ATTACH 'ducklake:/tmp/test_succeeds_appender_ducklake_via_temporary_table.ducklake' AS my_ducklake;
USE my_ducklake;
CREATE TABLE IF NOT EXISTS test (a INT);
""");

try (var appender = connection.createAppender("main", "test_temp")) {
appender.beginRow();
appender.append(124);
appender.endRow();
}

statement.execute("""
INSERT INTO test SELECT * FROM test_temp;
DROP TABLE test_temp;
""");

try (var rs = statement.executeQuery("select * from test")) {
rs.next();
then(rs.getInt(1)).isEqualTo(124);
}
}
}
}
```
Is there a plan to implement or fix it so that we can append directly the records to a DuckLake table?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.