duckdb / duckdb/duckdb-java

Error when appending to DuckLake table

Aperta
#249 7 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
C++
Stelle
127
Fork
80
Merge medio
13h 49m
PR unite (30g)
48

Descrizione

When I want to append many records to a DuckLake table with an appender, I get the following error when closing the `DuckDBAppender` instance. I used `duckdb-java` version 1.3.0.0.
```
java.sql.SQLException: INTERNAL Error: Calling GetStorage on a TableCatalogEntry that is not a DuckTableEntry
This error signals an assertion failure within DuckDB. This usually occurs due to unexpected conditions or errors in the program's logic.
For more information, see https://duckdb.org/docs/stable/dev/internal_errors
at org.duckdb.DuckDBNative.duckdb_jdbc_appender_close(Native Method)
at org.duckdb.DuckDBAppender.close(DuckDBAppender.java:102)
at test.DuckLakeAppenderTest.fails_appender_ducklake(DuckLakeAppenderTest.java:32)
at java.base/java.lang.reflect.Method.invoke(Method.java:565)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1604)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1604)
```

The following test cases may help to reproduce the error.
```java
package test;

import static org.assertj.core.api.BDDAssertions.*;

import java.sql.DriverManager;
import java.sql.SQLException;

import org.duckdb.DuckDBConnection;
import org.junit.jupiter.api.Test;

class DuckLakeAppenderTest {

@Test
void fails_appender_ducklake() throws SQLException {
try (
var connection = (DuckDBConnection) DriverManager.getConnection("jdbc:duckdb:");
var statement = connection.createStatement()
) {
// not necessary with duckdb 1.3.1
connection.setAutoCommit(false);

statement.execute("""
ATTACH 'ducklake:/tmp/test_fails_appender_ducklake.ducklake' AS my_ducklake;
USE my_ducklake;
CREATE TABLE IF NOT EXISTS test (a INT);
""");

try (var appender = connection.createAppender("main", "test")) {
appender.beginRow();
appender.append(124);
appender.endRow();
// fails: INTERNAL Error: Calling GetStorage on a TableCatalogEntry that is not a DuckTableEntry
}

try (var rs = statement.executeQuery("select * from test")) {
rs.next();
then(rs.getInt(1)).isEqualTo(124);
}
}
}

// workaround via temporary table
@Test
void succeeds_appender_ducklake_via_temporary_table() throws SQLException {
try (
var connection = (DuckDBConnection) DriverManager.getConnection("jdbc:duckdb:");
var statement = connection.createStatement()
) {
connection.setAutoCommit(false);

statement.execute("""
CREATE TEMP TABLE IF NOT EXISTS test_temp (a INT);
ATTACH 'ducklake:/tmp/test_succeeds_appender_ducklake_via_temporary_table.ducklake' AS my_ducklake;
USE my_ducklake;
CREATE TABLE IF NOT EXISTS test (a INT);
""");

try (var appender = connection.createAppender("main", "test_temp")) {
appender.beginRow();
appender.append(124);
appender.endRow();
}

statement.execute("""
INSERT INTO test SELECT * FROM test_temp;
DROP TABLE test_temp;
""");

try (var rs = statement.executeQuery("select * from test")) {
rs.next();
then(rs.getInt(1)).isEqualTo(124);
}
}
}
}
```
Is there a plan to implement or fix it so that we can append directly the records to a DuckLake table?

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Direzione di ricerca

Start with DuckLakeAppenderTest.java, especially fails_appender_ducklake, and run the reproduction against the reported version. Trace the close path shown in DuckDBAppender.java and the native appender close call. Done means appending directly to the DuckLake table closes without the internal error and the subsequent query returns 124.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
java, sql
Ambito
databases
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
45/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.