Facing unsafe memory access operation error while calling searcherManager.maybeReopen() [LUCENE-7527]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
We are getting below error while calling searcherManager.maybeReopen() method. We are using MMAP implementation to read NFS index directory mounted against 3 servers. We have a different process to update the indices and 3 other processes to read from the same index. What we believe is that this issue occurs when we call maybeReopen() method during heavy writes to the indices and MMap implementation is not able to coop with it..
Caused by: java.lang.InternalError: a fault occurred in a recent unsafe memory access operation in compiled Java code
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.lucene.store.MMapDirectory.cleanMapping(MMapDirectory.java:158)
at org.apache.lucene.store.MMapDirectory$MMapIndexInput.close(MMapDirectory.java:389)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:690)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:593)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:359)
at org.apache.lucene.index.SegmentInfos.readCurrentVersion(SegmentInfos.java:480)
at org.apache.lucene.index.DirectoryReader.isCurrent(DirectoryReader.java:901)
at org.apache.lucene.index.DirectoryReader.doOpenNoWriter(DirectoryReader.java:471)
at org.apache.lucene.index.DirectoryReader.doOpenIfChanged(DirectoryReader.java:450)
at org.apache.lucene.index.DirectoryReader.doOpenIfChanged(DirectoryReader.java:391)
at org.apache.lucene.index.IndexReader.openIfChanged(IndexReader.java:497)
at org.apache.lucene.search.SearcherManager.maybeReopen(SearcherManager.java:162)
---
Migrated from [LUCENE-7527](https://issues.apache.org/jira/browse/LUCENE-7527) by Jagmohan Singh, updated Nov 04 2016
Contributor guide
Research direction
Start with MMapDirectory.java, especially cleanMapping and MMapIndexInput.close, then trace the stack through DirectoryReader and SearcherManager.maybeReopen(). Reproduce the failure using an NFS-mounted index with MMAP access while another process performs heavy writes. Done means identifying the unsafe access cause and defining a verified correction or documented limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100