apache / apache/lucene

Good default weight for static scoring signals? [LUCENE-8218]

Open
#9,265 0 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Minor type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

On #9244, the question was raised whether we could come up with a good default value for the weight of a static scoring factor into the final score, which would make the functionality much easier to use. Currently it is 1.

One question that looks open for instance is whether these weights should be the same for all queries or not. Some papers said yes (they typically normalized query-dependent scores rather than scaling the weight based on the query-dependent scores, but this has the same effect in the end) while others, eg. the paper that #9244 is based on, just used a static weight for all queries. In both cases, optimal values for the weight were computed via training.

Another question is whether we should make the default weight depend on the similarity that is being used.

In the end, there is also a possibility that 1 is not a bad default at all. For instance if the weight of a term is log(x) where x is a fraction like df/docCount or ttf/sumTtf then it means that the static scoring factor has a weight that is the same as a term that appears in about 1/e \~ 37% of the corpus. In the particular case of BM25, it's actually closer to 1/(1+e) \~ 27%. The more I think about this issue, the more I'm erring on that side but I'd be curious to hear other opinions on this topic.

---
Migrated from [LUCENE-8218](https://issues.apache.org/jira/browse/LUCENE-8218) by Adrien Grand (@jpountz)

Contributor guide

Open the contributing guide

Research direction

Start by reviewing issue #9244 and the papers referenced in this discussion to understand the competing approaches to weighting static scoring factors. Done means reaching and documenting a justified decision about the default weight, including whether it should vary by query or similarity.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.