apache / apache/lucene

throw more specific exception on data corruption [LUCENE-8525]

Open
#9,571 12 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

DataInput throws generic IOException if data looks odd

[DataInput:141](https://github.com/apache/lucene-solr/blob/1d85cd783863f75cea133fb9c452302214165a4d/lucene/core/src/java/org/apache/lucene/store/DataInput.java#L141)

there are other examples like [BufferedIndexInput:219](https://github.com/apache/lucene-solr/blob/1d85cd783863f75cea133fb9c452302214165a4d/lucene/core/src/java/org/apache/lucene/store/BufferedIndexInput.java#L219), [CompressionMode:226](https://github.com/apache/lucene-solr/blob/1d85cd783863f75cea133fb9c452302214165a4d/lucene/core/src/java/org/apache/lucene/codecs/compressing/CompressionMode.java#L226) and maybe [DocIdsWriter:81](https://github.com/apache/lucene-solr/blob/1d85cd783863f75cea133fb9c452302214165a4d/lucene/core/src/java/org/apache/lucene/util/bkd/DocIdsWriter.java#L81)

That leads to some difficulties - see [elasticsearch #34322](https://github.com/elastic/elasticsearch/issues/34322)

It would be better if it throws more specific exception.

As a consequence [SegmentInfos.readCommit](https://github.com/apache/lucene-solr/blob/1d85cd783863f75cea133fb9c452302214165a4d/lucene/core/src/java/org/apache/lucene/index/SegmentInfos.java#L281) violates its own contract

```java
/**
* `@throws` CorruptIndexException if the index is corrupt
* `@throws` IOException if there is a low-level IO error
*/
```

---
Migrated from [LUCENE-8525](https://issues.apache.org/jira/browse/LUCENE-8525) by Vladimir Dolzhenko, updated Jan 11 2019

Contributor guide

Open the contributing guide

Research direction

Start by reading DataInput.java at line 141 and compare the corruption handling in BufferedIndexInput.java, CompressionMode.java, and DocIdsWriter.java. Check how SegmentInfos.readCommit handles these failures and use the linked Elasticsearch issue for context. Done means corruption paths raise a specific exception while low-level I/O errors remain IOException as documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.