Reconsider the best way to encode postings now that we can skip non-competitive hits [LUCENE-8760]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
The fact that we now skip non competitive hits has some implications to our postings:
- we are now more likely to call advance vs. nextDoc
- we are less likely to read term frequency for a given doc, since we only do that if the maximum score reported by impacts is competitive
- we are less likely to read positions for a given doc, since exact phrase queries first check the maximum score that would be obtained with a phrase freq equal to the minimum of all term freqs
It might be a good opportunity to re-explore the best way to encode postings.
---
Migrated from [LUCENE-8760](https://issues.apache.org/jira/browse/LUCENE-8760) by Adrien Grand (@jpountz)
Contributor guide
Research direction
No files or tests are named. Start by reviewing Lucene's postings encodings and the advance/nextDoc, term-frequency, positions, and impacts paths described in the issue. Done would require a decided encoding approach and evidence that it suits the changed access patterns.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100