apache / apache/lucene

QueryRescorer should be able to use the original sort on ties

Open
#14,455 7 comments 0 reactions 0 assignees View on GitHub
module:core/query/scoring type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

Currently the QueryRescorer uses the following logic for sorting the TopDocs after re-scoring:

```java
Comparator sortDocComparator =
(a, b) -> {
// Sort by score descending, then docID ascending:
if (a.score > b.score) {
return -1;
} else if (a.score < b.score) {
return 1;
} else {
// This subtraction can't overflow int
// because docIDs are >= 0:
return a.doc - b.doc;
}
};
```

The problem here is the `else` statement. So documents that have the same score will be stably sorted using their docIds, not their original sort order. So if users were using a complex sort (e.g. `score, price desc`) then re-scoring, the original sort (`price desc`) will not be used when there are ties in the new score.

This would be fixed by either removing the `return a.doc - b.doc`. Maybe even just changing the whole thing to `(a,b) -> b.score - a.score`.

If there is no original sort, then maybe keeping the original logic above makes sense. Both could be enabled by having QueryRescorer just accept an option to keep original order on ties.

Contributor guide

Open the contributing guide

Research direction

Start by locating QueryRescorer and its TopDocs sorting logic. Reproduce a tie between rescored documents with an original sort, then determine how the existing behavior should preserve that order and verify the result with the relevant QueryRescorer tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.