apache / apache/logging-log4j2
Memory leak in JdbcAppender when a database failure occurs during the commit/close phase.
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.7k
- Avg merge
- 21h 30m
- Merged PRs (30d)
- 27
Description
A memory leak occurs when using the JdbcAppender with a configured bufferSize. When the appender attempts to flush the buffer, it calls AbstractDatabaseManager.flush(), which in turn executes the SQL statements and calls commitAndClose().
If commitAndClose() throws an exception (e.g., due to a temporary database disconnection, transaction timeout, or constraint violation), the exception propagates out of the flush() method before buffer.clear() is executed. This results in:
1. Duplicate Events: The events remain in the buffer and will be processed again in the next flush attempt.
2. Memory Exhaustion (OOM): New events continue to be added to the internal buffer (or the buffer never gets a chance to reset its state), eventually leading to an OutOfMemoryError as the application continues to log.
Steps to Reproduce
1. Configure a JdbcAppender in log4j2.xml with a bufferSize (e.g., bufferSize="100").
2. Start the application and ensure it logs to the database correctly.
3. Induce a database failure (e.g., stop the database service or revoke the user's insert permissions).
4. Generate enough log events to trigger multiple buffer flushes.
5. Observe via a heap dump or memory profiler that the ArrayList (or equivalent buffer) inside the DatabaseManager continues to hold onto LogEvent objects and grows indefinitely if not cleared, or simply keeps the same failing events while the appender fails to recover.
Suggested Fix
The flush() method in AbstractDatabaseManager should ensure that the buffer is cleared even if an exception occurs during the database operations. A try-finally block should be used:
```
public synchronized void flush() {
if (this.buffer.size() > 0) {
try {
this.connectAndStart();
for (final LogEvent event : this.buffer) {
this.writeInternal(event, this.layout.toSerializable(event));
}
this.commitAndClose();
} finally {
// Ensure the buffer is cleared to prevent OOM and duplicate processing attempts
this.buffer.clear();
}
}
}
```
Alternatively, JdbcDatabaseManager.commitAndClose() should catch and log exceptions internally (as implemented in the local CustomJdbcDatabaseManager workaround) to prevent them from interrupting the manager's lifecycle.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with AbstractDatabaseManager.flush(), which is called while JdbcAppender flushes its configured buffer, and inspect how JdbcDatabaseManager.commitAndClose() handles failures. The fix is done when a commit or close exception cannot leave LogEvent objects retained for repeated processing or unbounded buffer growth.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100