duckdb / duckdb/duckdb-java

FATAL Error: Failed to create checkpoint because of error...

Open
#206 18 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
127
Forks
80
Avg merge
13h 49m
Merged PRs (30d)
48

Description

JAVA 17, DuckDB 1.2 (duckdb_jdbc-1.2.1.jar)

I am trying to perform some queued INSERTs (specifically preparedStatement, batch inserts) using CompletableFuture (i.e. background executor, but same process). All these go to the same table.

The background method uses (partially) :
`try (var connection = duckDBConnection.duplicate();
var statement = connection.prepareStatement(INSERT_TEMPLATE))) {
for (...) {
statement.setString();
statement.set...
statement.addBatch();
...
}
statement.executeBatch();
} `

I'm using CompletableFuture.runAsync, so these go on the Java commonPool() executor, queued to run asap.

After a few of these complete, I'm getting the exception:
`java.sql.SQLException:` FATAL Error: Failed to create checkpoint because of error: Failed to delete file "e:\vectorindex\efr.duckdb.wal": The process cannot access the file because it is being used by another process.

at org.duckdb.DuckDBNative.duckdb_jdbc_execute(Native Method)
at org.duckdb.DuckDBPreparedStatement.execute(DuckDBPreparedStatement.java:148)
at org.duckdb.DuckDBPreparedStatement.executeBatchedPreparedStatement(DuckDBPreparedStatement.java:489)
at org.duckdb.DuckDBPreparedStatement.executeBatch(DuckDBPreparedStatement.java:474)
...`

This is entirely unexpected since all of the code is running in the same process, and according to the docs, this should be fine. And they are all unique INSERTs (there isn't even a PK on the table), so there should be no row-based table contention.

It appears that DuckDB is performing checkpoints on its own...maybe when the duplicate session is closing in the method? And whether it is performing checkpoints or not, this appears to be a bug, because it's documented that multiple threads within the same process should be able to r/w the same database.

Does the WAL strategy need to be adjusted (internally)?

Are there any workarounds for now?

Thanks!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.