apache / apache/lucene

IBSimilarity causes test failure in TestSynonymQuery

Open
#16,137 4 comments 0 reactions 0 assignees View on GitHub
type:test
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

`IBSimilarity` with `DistributionSPL` can produce negative scores for very large tf values, which `TestSynonymQuery.testBoosts()` uses. We don't have a good way to suppress individual Similarity implementations for tests at the moment, but I think in any case we should be looking to fix the root Similarity itself? Impacts and competitive iterators all assume that scores can't be negative, a contract which this Similarity breaks.

### Gradle command to reproduce

On `branch_10x`:
```
./gradlew :lucene:core:test --tests "org.apache.lucene.search.TestSynonymQuery.testBoosts" -Ptests.jvms=12 -Ptests.jvmargs= -Ptests.seed=EC3CC718CF767FE7 -Ptests.useSecurityManager=false -Ptests.gui=true
-Ptests.file.encoding=US-ASCII -Ptests.vectorsize=default -Ptests.forceintegervectors=false
```

Contributor guide

Open the contributing guide

Research direction

Start by running the provided Gradle command for lucene:core and TestSynonymQuery.testBoosts(). Then inspect IBSimilarity with DistributionSPL and the scoring path used by the test. Done means the Similarity no longer produces negative scores for the large term-frequency values involved and the reproducing test passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.