apache / apache/lucene

Support multiple ToParentBlockJoinQuery instances in a BooleanQuery with Occur == MUST/FILTER [LUCENE-10056]

Open
#11,094 0 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Minor module:join type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

As a user of `ToParentBlockJoinQuery` (TPBJQ), we've run into some fairly tricky cases where we'd like to include multiple instances in a `BooleanQuery` tree that are all required (i.e., MUST/FILTER). The only way to get this to work properly today is to ensure all constraints are packed into a single TPBJQ, otherwise it may produce incorrect matches.

To illustrate with an example, consider child documents that contain a "price" (numeric value) and "condition" ("new" vs. "used") field. Consider constructing a query that matches these child documents if "price < 100" and "condition == new". If one TPBJQ is constructed with a `childQuery` modeling both constraints, this works just fine and will provide parent docs that contain at least one child matching both constraints. But, if we were to create two separate TPBJQ instances, one for "price" and one for "condition" and then try to conjunctively join those two instances, we'd get parent documents that contained at least one "condition == new" child and one "price < 100" child but those might occur on different children and the parent may actually be a false match.

While there's nothing inherently defective with TPBJQ, this feels a bit trappy and easy to get wrong. Even trickier, there are cases where it's not possible to pack all conjunctive child constraints into a single TPBJQ (e.g., using `DrillSideways` faceting where the query needs to be iteratively/partially evaluated).

It would be nice to come up with a solution that allows multiple TPBJQ instances to exist in a BooleanQuery, maybe doing some later-stage filtering to ensure the parents are actually proper matches.

---
Migrated from [LUCENE-10056](https://issues.apache.org/jira/browse/LUCENE-10056) by Greg Miller (@gsmiller)

Contributor guide

Open the contributing guide

Research direction

Start by reading the ToParentBlockJoinQuery and BooleanQuery entry points to understand how multiple required joins are currently combined. Define and validate a design that prevents false parent matches when separate TPBJQ instances constrain different children, including the DrillSideways case; done means conjunctive joins return only parents with a single child satisfying all constraints.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.