apache / apache/maven-indexer

Memory Leak on RHEL 8: JVM Retains File Handles After Index Deletion

Open
#702 5 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
74
Forks
69
Avg merge
14h 8m
Merged PRs (30d)
6

Description

### Affected version

7.1.6

### Bug description

Problem Description
We are experiencing a significant memory leak that appears to be specific to the RHEL 8 environment. During our indexing process, we create and then subsequently delete a Lucene index using maven-indexer. Although the index directory and its files are successfully removed from the filesystem, the JVM process continues to hold onto the file handles, preventing memory from being released.

Steps to Reproduce
Create a new indexing context using org.apache.maven.index.DefaultIndexer#createIndexingContext(...).

Perform indexing operations, which trigger org.apache.lucene.index.DocumentsWriter#updateDocuments and create compound segment files (.cfs).

After indexing is complete, close the context and request deletion of the index files by calling org.apache.maven.index.DefaultIndexer#closeIndexingContext(context, true).

Observe that the index directory is deleted from the filesystem as expected.

Monitor the JVM's memory and open file descriptors.

Expected Behavior
The JVM should release the file handles for the deleted .cfs files, and the associated memory should be garbage collected.

Actual Behavior
The JVM retains the file handles for the deleted index files, resulting in a continuous increase in memory consumption. This issue is only reproducible on RHEL 8.

Image

Version and environment details
Lucene version - 9.12.2
Maven indexer version - 7.1.6
JVM - 21
OS - Red Hat Enterprise Linux 8

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with DefaultIndexer#createIndexingContext and DefaultIndexer#closeIndexingContext(context, true), then reproduce the indexing and deletion flow on RHEL 8 with Lucene 9.12.2. Inspect the file-handle lifecycle around DocumentsWriter#updateDocuments and compound .cfs files. Done means deleted index files no longer remain open and memory and file-descriptor usage stop increasing.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.