Monitor alternative "TermFilter" Presearcher for sparse filter fields [LUCENE-10571]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
One of the things that surprised me the most when looking into how the `TermFilteredPresearcher` worked was what happens when Queries and/or Documents do _NOT_ have a value in a configured filter field.
per the javadocs...
> Filtering by additional fields can be configured by passing a set of field names\. Documents that contain values in those fields will only be checked against \{`@link` MonitorQuery} instances that have the same fieldname-value mapping in their metadata\.
...which is straightforward and useful in the tested example where every registered Query has `"language"` metadata, and every Document has a `"language"` field, but gives unintuitive results when a Query or Document does **NOT** have a `"language"`
A more "intuitive" & useful (in my opinions) implementation would be something that could be documented as ...
> Filtering by additional fields can be configured by passing a set of field names\. Documents that contain values in those fields will only be checked against \{`@link` MonitorQuery} instances
> that have the same fieldname-value mapping in their metadata <em>or have no mapping for that fieldname</em>\.
>
> Documents that do not contain values in those fields will only be checked against \{`@link` MonitorQuery} instances that also have no mapping for that fieldname\.
...ie: instead of being a straight "filter candidate queries by what we find in the filter fields in the documents" we can instead "derive the queries that are viable candidates for each document if we were restricting the set of documents by those values during a "forward search"
---
Migrated from [LUCENE-10571](https://issues.apache.org/jira/browse/LUCENE-10571) by Chris M. Hostetter (@hossman), updated Jun 27 2022
Attachments: [LUCENE-10571.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-10571/LUCENE-10571.patch)
Contributor guide
Research direction
Start with the issue description and the archived LUCENE-10571.patch, then inspect the TermFilteredPresearcher behavior described in the javadocs. Establish how missing filter values currently behave and define tests covering absent query and document metadata. Done means the intended candidate-selection behavior is implemented and documented with passing tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100