Codec classes/packages that do not provide (automatic) file format back should throw an explicit exception [LUCENE-5126]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
If a user builds an index with an alternate codec that does not provide (automatic) backcompat for reads in future versions of lucene, the resulting error if the user tries to upgrade can be obscure and missleading – suggesting either index corruption or a code bug w/o any obvious indicator that the real problem is that the user needs to explicitly convert the index format. (Even if we document the hell out of this – #6189 – so users have to know about it when they pick a codec, the person who picked the codec for an app may be long gone when some other poor sap tries to upgrade that app to use a newer version of Lucene)
For example, as demonstrated in #6185 if a user uses "Disk" based doc values in 4.2.1, and then tries upgrading to 4.4, they might get...
```
java.io.EOFException: read past EOF: MMapIndexInput(path="/home/hossman/tmp/disk_docvals_bug/collection1/data/index/_0_Disk_0.dvdm")
```
or they might get...
```
org.apache.lucene.index.CorruptIndexException: invalid type: 65, resource=MMapIndexInput(path="/storage/solrdata/index/_fd_Disk_0.dvdm")
```
...it would be a lot better for our users if in situations like this we could give them a clear and definitive indication (similar to "IndexFormatTooOldException") that the codec they are using has changed in a non-backcompat way, and that they need to explicitly convert their index to supported codec before upgrading.
---
Migrated from [LUCENE-5126](https://issues.apache.org/jira/browse/LUCENE-5126) by Chris M. Hostetter (@hossman)
Contributor guide
Research direction
No source files or tests are named. Start by tracing how alternate codecs are read during index upgrades, using the Disk doc-values failures in issue #6185 and the related documentation issue #6189 as context. Done means a clear, definitive exception identifies non-backward-compatible codec changes and tells users to convert the index before upgrading.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100