Enforcing BooleanClause.Occur.FILTER before CustomScoreQuery [LUCENE-7443]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
I have a BooleanQuery including a BooleanClause.Occur.MUST clause and a BooleanClause.Occur.FILTER clause. I then use the BooleanQuery in a CustomScoreQuery. It seems that the filtering constraint is not enforced before calculating the score in CustomScoreQuery and is enforced after the calculation. This can cause performance issues, since CustomScoreQuery may perform costly operations to rescore the BooleanQuery results. If the filter is enforced before rescoring, much less documents may require rescoring in CustomScoreQuery, and it does not make sense to spend a lot of processing for documents we know will be filtered out.
More details:
The BooleanQuery is used as subQuery in the following constructor, and a costly scoringQuery is used which is more rational to calculate it only when the results of subQuery is not trivially filtered out because of the FILTER in subQuery.
public CustomScoreQuery(Query subQuery, FunctionQuery scoringQuery) {
---
Migrated from [LUCENE-7443](https://issues.apache.org/jira/browse/LUCENE-7443) by Shayan Tabrizi
Contributor guide
Research direction
Start at the CustomScoreQuery(Query subQuery, FunctionQuery scoringQuery) constructor and trace how its BooleanQuery subQuery handles MUST and FILTER clauses. Determine where filtering occurs relative to the costly scoringQuery calculation. Done means FILTER constraints are applied before rescoring, avoiding work for documents that will be filtered out.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100