apache / apache/lucene

Cutover to Java7 nio APIS to improve SegmentInfos fallback logic [LUCENE-5910]

Open
#6,972 0 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 6h
Merged PRs (30d)
86

Description

Spinoff from #6966 ...

Today we have crazy logic (SegmentInfos.FindSegmentsFile) to try to locate the "latest" segments_N file from the index directory, even while a writer commits (writes a new segments_N+1 and removes the old one) concurrently while we try to open a DirectoryReader on the index.

But it's scary because it catches any IOException and assumes that segments_N was no good and tries to find an older one to load.

Really it should be more picky, and only fall back e.g. if it hit EOFE, or if one of the SegmentReaders hit FNFE, and for other exceptions e.g. transient problems like "out of file desctriptors" or OOME or "access denied" it should immediately throw these back to caller instead of falling back which risks losing documents in the index.

---
Migrated from [LUCENE-5910](https://issues.apache.org/jira/browse/LUCENE-5910) by Michael McCandless (@mikemccand)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.