[Bug] Postings force merge regression between Lucene 9.12 and Lucene 10.0
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
This is fork of the issue https://github.com/apache/lucene/issues/14463 specific to postings format regression.
As part of 10.0 Lucene, default readAdvice changed to random and we noticed that termsIndex which uses default readAdvice regressed during force merge.
```
String termsName =
IndexFileNames.segmentFileName(segment, state.segmentSuffix, TERMS_EXTENSION);
termsIn = state.directory.openInput(termsName, state.context);
```
So there could be other implementations and file formats which has similar regressions but fixing the `termsIn` removed the 20% regression we saw between OpenSearch 2.19 [ Lucene 9.12.1] and OpenSearch 3.0 [Lucene 10.1].
So creating this issue to track postings related force merge regressions and fixes.
### Version and environment details
_No response_
Contributor guide
Research direction
Start by tracing the postings implementations involved in force merge, especially the termsIndex path and the shown IndexFileNames.segmentFileName/openInput entry points. Compare behavior between Lucene 9.12 and 10.0, then use force-merge measurements to identify and verify any remaining postings-format regression fixes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- performance, search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100