apache / apache/lucene

Facing unsafe memory access operation error while calling searcherManager.maybeReopen() [LUCENE-7527]

Open
#8,578 3 comments 0 reactions 0 assignees View on GitHub
affects-version:3.5 legacy-jira-priority:Major module:core/search type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

We are getting below error while calling searcherManager.maybeReopen() method. We are using MMAP implementation to read NFS index directory mounted against 3 servers. We have a different process to update the indices and 3 other processes to read from the same index. What we believe is that this issue occurs when we call maybeReopen() method during heavy writes to the indices and MMap implementation is not able to coop with it..

Caused by: java.lang.InternalError: a fault occurred in a recent unsafe memory access operation in compiled Java code
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.lucene.store.MMapDirectory.cleanMapping(MMapDirectory.java:158)
at org.apache.lucene.store.MMapDirectory$MMapIndexInput.close(MMapDirectory.java:389)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:690)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:593)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:359)
at org.apache.lucene.index.SegmentInfos.readCurrentVersion(SegmentInfos.java:480)
at org.apache.lucene.index.DirectoryReader.isCurrent(DirectoryReader.java:901)
at org.apache.lucene.index.DirectoryReader.doOpenNoWriter(DirectoryReader.java:471)
at org.apache.lucene.index.DirectoryReader.doOpenIfChanged(DirectoryReader.java:450)
at org.apache.lucene.index.DirectoryReader.doOpenIfChanged(DirectoryReader.java:391)
at org.apache.lucene.index.IndexReader.openIfChanged(IndexReader.java:497)
at org.apache.lucene.search.SearcherManager.maybeReopen(SearcherManager.java:162)

---
Migrated from [LUCENE-7527](https://issues.apache.org/jira/browse/LUCENE-7527) by Jagmohan Singh, updated Nov 04 2016

Contributor guide

Open the contributing guide

Research direction

Start with MMapDirectory.java, especially cleanMapping and MMapIndexInput.close, then trace the stack through DirectoryReader and SearcherManager.maybeReopen(). Reproduce the failure using an NFS-mounted index with MMAP access while another process performs heavy writes. Done means identifying the unsafe access cause and defining a verified correction or documented limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.