Strengthen checks on allowed segment file names [LUCENE-6353]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
Replication layers that need to move Lucene indices around frequently need to (or should) be able to decode an given index file into its segment name, segment suffix (generation and/or per-field "suffix"), file name extension.
This is useful for robustness, e.g. copying all of a segment's files if any checksums/ids across those files disagrees between source and replica.
But we don't do very strong checking of the "syntax" of an index file name today, or at least we should do the TODO in SegmentWriteState (upgrade the assert on the segment suffix to a real check).
I think we could also make MDW.close insist every file it finds in a closed index either matches the required syntax, or is an "extra" added by ExtraFS?
---
Migrated from [LUCENE-6353](https://issues.apache.org/jira/browse/LUCENE-6353) by Michael McCandless (@mikemccand), updated May 09 2016
Contributor guide
Research direction
Start with the TODO in SegmentWriteState and inspect MDW.close, focusing on how segment names, suffixes, generations, and extensions are validated. Determine the required filename syntax and how ExtraFS marks additional files; done means closed-index files are rejected unless they match the syntax or are valid extras.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100