FATAL Error: Database invalidated. Failed to create checkpoint as .wal file does not exist
- Dominant language
- C++
- Stars
- 127
- Forks
- 80
- Avg merge
- 13h 49m
- Merged PRs (30d)
- 48
Description
Java 21, Duckdb 1.2.2.0, OS: Linux Redhat 10
In my Java application, I am opening multiple connections from different threads to Duckdb in persistent mode. In the start of the application, I create a main connection, and now any subsequent connection is created using `DuckDBConnection.duplicate`.
DuckDB file is only accessed from this process.
My connection is created with property `jdbc_stream_results` set to true.
One of my update query failed with error:
```
java.sql.SQLException: FATAL Error: Failed: database has been invalidated because of a previous fatal error. The database must be restarted prior to being used again.
Original error: "Failed to create checkpoint because of error: Could not remove file "/home/ec2-user/duckdb/./duckdb.db.wal": No such file or directory"
```
Scenario:
1. DuckDb is being queried by a select query on a different table parallely.
2. Update/INSERT queries are coming. They are processed in different threads. To run a query, I create new connection (using duplicate) and run the query.
My duckdb.db file is around 160 GB.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the concurrent select and update/INSERT scenario using DuckDBConnection.duplicate with jdbc_stream_results enabled, focusing on the checkpoint failure involving duckdb.db.wal. Inspect the Java JDBC connection behavior around duplicate connections and determine whether the database remains valid during concurrent access without a missing-WAL-file error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, linux, sql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100