PassageScorer could take proximity of terms into account [LUCENE-10011]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
The UnifiedHighlighter scores its highlighted passages using a modified term frequency calculation, similar to BM25. This means that two passages containing the same set of terms will score equivalently. Given that proximity is often a reasonable proxy for relevance, and that passages contain the offsets of their internal hits, it would be useful to add the option of also weighting by the difference between the start of the first hit and the end of the last within the passage.
---
Migrated from [LUCENE-10011](https://issues.apache.org/jira/browse/LUCENE-10011) by Alan Woodward (@romseygeek)
Contributor guide
Research direction
Start at the UnifiedHighlighter's PassageScorer and inspect how passage hit offsets are exposed. Determine how an optional proximity weighting based on the first hit's start and last hit's end should interact with the existing scoring. Done means passages with different term spans can receive different scores when the option is enabled, while the existing behavior remains available.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100