DirectoryReader.listCommits graceful handling of corrupt commits is broken [LUCENE-7843]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
DirectoryReader.listCommits expects FileNotFoundException and NoSuchFileException while trying to read a specific commit file - if these are thrown then the code handles this gracefully and that specific commit file is not returned in the list of all commits for that directory.
#8713 changed the behavior of SegmentInfos.readCommit to not throw these exceptions anymore, but rather throw CorruptIndexException instead.
This causes the code in DirectoryReader.listCommits to never encounter the previous two exception types, thus throwing the CorruptIndexException all the way to the top, instead of gracefully handling a few missing commit files.
This breaks how Luke does a clean index operation when listing all of the commits and then proceeding to delete any files inside the index directory that were not mentioned by any of the commits.
---
Migrated from [LUCENE-7843](https://issues.apache.org/jira/browse/LUCENE-7843) by Rei Shapira
Environment:
```
Windows, MMapDirectory
```
Linked issues:
- #8713
Contributor guide
Research direction
Start with DirectoryReader.listCommits and inspect how it handles failures from SegmentInfos.readCommit, using linked issue #8713 for the behavior change. Confirm the clean-index scenario where missing or corrupt commit files are skipped rather than causing CorruptIndexException to escape, and add or update the relevant regression coverage if the surrounding code identifies it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100