apache / apache/lucene

Use BoostAttribute in in TokenFilters to denote Terms that QueryParser should give lower boosts [LUCENE-3130]

Open
#4,203 15 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major module:core/queryparser type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

A recent thread asked if there was anyway to use QueryTime synonyms such that matches on the original term specified by the user would score higher then matches on the synonym. It occurred to me later that a float Attribute could be set by the SynonymFilter in such situations, and QueryParser could use that float as a boost in the resulting Query. IThis would be fairly straightforward for the simple "synonyms => BooleamQuery" case, but we'd have to decide how to handle the case of synonyms with multiple terms that produce MTPQ, possibly just punt for now)

Likewise, there may be other TokenFilters that "inject" artificial tokens at query time where it also might make sense to have a reduced "boost" factor...

- SynonymFilter
- CommonGramsFilter
- WordDelimiterFilter
- etc...

In all of these cases, the amount of the "boost" could me configured, and for back compact could default to "1.0" (or null to not set a boost at all)

Furthermore: if we add a new BoostAttrToPayloadAttrFilter that just copied the boost attribute into the payload attribute, these same filters could give "penalizing" payloads to terms when used at index time) could give "penalizing" payloads to terms.

---
Migrated from [LUCENE-3130](https://issues.apache.org/jira/browse/LUCENE-3130) by Chris M. Hostetter (@hossman), 1 vote, updated Feb 27 2015
Linked issues:
- [SOLR-5379](https://issues.apache.org/jira/browse/SOLR-5379)
- #2898

Contributor guide

Open the contributing guide

Research direction

Start by reviewing SynonymFilter, CommonGramsFilter, WordDelimiterFilter, and QueryParser to understand how injected query-time tokens are represented. The scope still requires a decision about multi-term synonyms and whether BoostAttribute should also be copied into payloads; done would mean an agreed, implemented behavior for these cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
18/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.