Can IW sort segments on flush too? [LUCENE-7281]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
Spinoff from #7824, where we made index-time sorting first class in Lucene, except only merged segments get sorted.
Can we also sort flushed segments? This would be cleaner because then search-time users don't need to fret about whether a given segment is sorted or not, and it would also improve the current "best effort" check IW does that you didn't try to change the index sort order, to an accurate check.
But this is tricky because e.g. at least term vectors and stored fields write "live" to the segment's (codec's) files as documents are being indexed...
---
Migrated from [LUCENE-7281](https://issues.apache.org/jira/browse/LUCENE-7281) by Michael McCandless (@mikemccand)
Contributor guide
Research direction
Start by studying the existing index-time sorting and segment-merge behavior referenced in the issue, then trace how flushed segments write term vectors and stored fields directly through codec files. Done means flushed segments are sorted as well as merged segments, and the index-sort check accurately detects attempts to change the index sort order.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100