Reduce memory usage of skip reader
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
Following up on #13576. Currently, we always use the size `Lucene99PostingsFormat.MAX_SKIP_LEVELS` to initialize the skip reader.
https://github.com/apache/lucene/blob/30c965ea575a5e75c2bf724a340aa690d82f1ec5/lucene/core/src/java/org/apache/lucene/codecs/lucene99/Lucene99PostingsReader.java#L496-L501
We can take the max docs into consideration to calculate the maximum number of skip levels like we did in `MultiLevelSkipListWriter`.
https://github.com/apache/lucene/blob/30c965ea575a5e75c2bf724a340aa690d82f1ec5/lucene/core/src/java/org/apache/lucene/codecs/MultiLevelSkipListWriter.java#L75C8-L82
Contributor guide
Research direction
Start in Lucene99PostingsReader.java around the skip-reader initialization linked in the issue, then compare the maximum-level calculation in MultiLevelSkipListWriter.java. Confirm that the reader's allocation accounts for the maximum document count rather than always using Lucene99PostingsFormat.MAX_SKIP_LEVELS, and verify the relevant Lucene tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Refactor
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100