Optimization for querying against an index sorted in the opposite order
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
If an index with very large doc count, is sorted: `myfield:asc`, and the query has sort spec: `myfield:asc`, we can get early-termination goodness and opens doors for many other optimization opportunities.
However, when query has sort spec: `myfield:desc`, this presents to be a worst case scenario because we would be `up-heap` and `rebalance` for every hit, because larger docids will always win the competition against the min-val of the priority queue.
The proposal is to catch this case, and use a FIFO queue like `ArrayDequeue` instead of `PriorityQueue` to always insert into the queue.
I am not sure if this is too much of an edge case to be included in Lucene, feel free to close the issue if so.
Contributor guide
Research direction
No file or test is named. Start by tracing the search path that uses PriorityQueue for an index sorted opposite to the query sort, then assess the proposed FIFO queue approach for this case. Done should mean the opposite-order query avoids per-hit heap rebalancing without changing its results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100