apache / apache/lucene

Change in behavior using SimpleCollector+TopScoreDocCollector between 10.2 and 10.3 when scoreMode==COMPLETE

Open
#15,239 8 comments 0 reactions 0 assignees View on GitHub
type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

Attempting to upgrade Solr to use Lucene 10.3 caused a regression in one of our tests, which i tracked down a to a change in behavior when using a SimpleCollector (subclass) that wraps a TopScoreDocCollector instance

It's not clear to me if this new 10.3 behavior is technically "wrong", or if Solr is (miss|ab)using some of the Collector APIs -- so i wanted to file this report for completeness (Even if Solr ultimately makes other changes to work around this change in behavior)

----

I've boiled the problem down to the simplest test case patch i can think of that demonstrates the problem. In 10.2, both asserts in this test pass, in 10.3, neither assert will pass.

The crux of the issue as i understand it, is that TopScoreDocCollector tracks competitive scoring info that it cares about in the Scorable, and that info is (now) being used by the Scorer to limit what documents are collected -- even though the Collector says it wants ScoreMode.COMPLETE collection.

[collector_behavior_change_10_3.patch.txt](https://github.com/user-attachments/files/22568261/collector_behavior_change_10_3.patch.txt)

(FWIW: the patch modifies TestMultiCollector simply because i didn't see a more obvious place to add tests of subclassing SimpleCollector)

### Version and environment details

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with the attached collector_behavior_change_10_3.patch.txt and the TestMultiCollector tests it modifies. Reproduce the two failing assertions on Lucene 10.3 and compare them with 10.2, then inspect the SimpleCollector, TopScoreDocCollector, Scorable, and ScoreMode interactions described in the report. Done means the behavior is understood and the agreed semantics are covered by a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.