apache / apache/lucene

TopTermsBlendedFreqScoringRewrite should use SynonymQuery [LUCENE-8840]

Open
#9,884 4 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Today the TopTermsBlendedFreqScoringRewrite, which is the default rewrite method for Fuzzy queries, uses the BlendedTermQuery to score documents that match the fuzzy terms. This query blends the frequencies used for scoring across the terms and creates a disjunction of all the blended terms. This means that each fuzzy term that match in a document will add their BM25 score contribution. We already have a query that can blend the statistics of multiple terms in a single scorer that sums the doc frequencies rather than the entire BM25 score: the SynonymQuery. Since #9698 this query also handles boost between 0 and 1 so it should be easy to change the default rewrite method for Fuzzy queries to use it instead of the BlendedTermQuery. This would bound the contribution of each term to the final score which seems a better alternative in terms of relevancy than the current solution.

---
Migrated from [LUCENE-8840](https://issues.apache.org/jira/browse/LUCENE-8840) by Jim Ferenczi (@jimczi), 1 vote, updated Jun 12 2019
Attachments: [LUCENE-8840.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-8840/LUCENE-8840.patch)

Contributor guide

Open the contributing guide

Research direction

Start with TopTermsBlendedFreqScoringRewrite and the default rewrite path for Fuzzy queries, then compare its use of BlendedTermQuery with SynonymQuery’s boost handling described in the issue. Identify the relevant Lucene scoring and rewrite tests before changing the default; done means fuzzy-query scoring uses SynonymQuery and the affected tests pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.