apache / apache/lucene

DirectoryReader.listCommits graceful handling of corrupt commits is broken [LUCENE-7843]

Open
#8,894 0 comments 0 reactions 0 assignees View on GitHub
affects-version:6.5.1 legacy-jira-priority:Major module:core/index type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

DirectoryReader.listCommits expects FileNotFoundException and NoSuchFileException while trying to read a specific commit file - if these are thrown then the code handles this gracefully and that specific commit file is not returned in the list of all commits for that directory.

#8713 changed the behavior of SegmentInfos.readCommit to not throw these exceptions anymore, but rather throw CorruptIndexException instead.

This causes the code in DirectoryReader.listCommits to never encounter the previous two exception types, thus throwing the CorruptIndexException all the way to the top, instead of gracefully handling a few missing commit files.

This breaks how Luke does a clean index operation when listing all of the commits and then proceeding to delete any files inside the index directory that were not mentioned by any of the commits.

---
Migrated from [LUCENE-7843](https://issues.apache.org/jira/browse/LUCENE-7843) by Rei Shapira
Environment:
```
Windows, MMapDirectory
```

Linked issues:
- #8713

Contributor guide

Open the contributing guide

Research direction

Start with DirectoryReader.listCommits and inspect how it handles failures from SegmentInfos.readCommit, using linked issue #8713 for the behavior change. Confirm the clean-index scenario where missing or corrupt commit files are skipped rather than causing CorruptIndexException to escape, and add or update the relevant regression coverage if the surrounding code identifies it.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.