apache / apache/lucene

Memory Leak on RHEL 8: JVM Retains File Handles After Index Deletion

Open
#15,262 9 comments 0 reactions 1 assignee Claimed by @uschindler View on GitHub
type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

**Problem Description**
We are experiencing a significant memory leak that appears to be specific to the RHEL 8 environment. During our indexing process, we create and then subsequently delete a Lucene index using maven-indexer. Although the index directory and its files are successfully removed from the filesystem, the JVM process continues to hold onto the file handles, preventing memory from being released.

**Steps to Reproduce**
Create a new indexing context using org.apache.maven.index.DefaultIndexer#createIndexingContext(...).

Perform indexing operations, which trigger org.apache.lucene.index.DocumentsWriter#updateDocuments and create compound segment files (.cfs).

After indexing is complete, close the context and request deletion of the index files by calling org.apache.maven.index.DefaultIndexer#closeIndexingContext(context, true).

Observe that the index directory is deleted from the filesystem as expected.

Monitor the JVM's memory and open file descriptors.

**Expected Behavior**
The JVM should release the file handles for the deleted .cfs files, and the associated memory should be garbage collected.

**Actual Behavior**
The JVM retains the file handles for the deleted index files, resulting in a continuous increase in memory consumption. This issue is only reproducible on RHEL 8.

Image

### Version and environment details

Lucene version - 9.12.2
Maven indexer version - 7.1.6
JVM - 21
OS - Red Hat Enterprise Linux 8

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.