apache / apache/lucene

FieldComparators Should Not Maintain Implicit PQs [LUCENE-8950]

Open
#9,993 5 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

While doing some perf tests, I realised that FieldComparators inherently maintain implicit priority queues for maintaining the sorted order of documents for the given sort order. This is wasteful especially in the case of a multi feature sort order and a large number of hits requested.

 

We should change this to have FieldComparators maintain only the top and bottom values, and use them as barriers to compare

---
Migrated from [LUCENE-8950](https://issues.apache.org/jira/browse/LUCENE-8950) by Atri Sharma (@atris)

Contributor guide

Open the contributing guide

Research direction

Start by locating the FieldComparators implementation and reading how it maintains implicit priority queues for sorted hits. Review the existing performance tests mentioned in the issue, if present. Done means FieldComparators maintain only top and bottom values and use them as comparison barriers without changing the required sort behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.