Memory leak in long running transaction
- Dominant language
- Kotlin
- Stars
- 9.3k
- Forks
- 798
- Avg merge
- 4d 2h
- Merged PRs (30d)
- 26
Description
Hi,
Using pgsql & suspended transactions in dsl mode, we have a long running transaction with a loop, e.g.
```kotlin
transaction {
var doContinue = true
while (doContinue) {
val toProcess = getDbElementsToProcess()
processElements(toProcess)
if (toProcess.isEmpty()) doContinue = false
}
}
```
If we have a huge number of elements to process (> 100 000) and the transactions lasts few minutes, we can eventually have an Out Of Memory. Looking in VisualVM, we see this is caused by multiple PreparedStatements that are never garbage collected.
Yet if we open a new transaction on each loop run, we don't have troubles anymore. e.g.
```kotlin
var doContinue = true
while (doContinue) {
transaction {
val toProcess = getDbElementsToProcess()
processElements(toProcess)
if (toProcess.isEmpty()) doContinue = false
}
}
```
From another post, we understood that closing the transaction does close all prepared statements and free the memory.
However we'd like to avoid having to open a new transaction on each loop run. Is there any way to close prepared statements properly, optionally manually?
Contributor guide
Research direction
Start with the suspended-transaction DSL and the PreparedStatement lifecycle described in the report, using the provided long-running loop to reproduce the leak. Use VisualVM to confirm statement retention, then verify that prepared statements are released without requiring a new transaction for every loop iteration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kotlin, postgresql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100