Cascaded grouping [LUCENE-3122]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
Similar to [SOLR-2526](https://issues.apache.org/jira/browse/SOLR-2526), in that you are grouping on 2 separate fields, but instead of treating those fields as a single grouping by a compound key, this change would let you first group on key1 for the primary groups and then secondarily on key2 within the primary groups.
Ie, the result you get back would have groups A, B, C (grouped by key1) but then the documents within group A would be grouped by key 2.
I think this will be important for apps whose documents are the product of denormalizing, ie where the Lucene document is really a sub-document of a different identifier field. Borrowing an example from #4170, you have doctors but each doctor may have multiple offices (addresses) where they practice and so you index doctor X address as your lucene documents. In this case, your "identifier" field (that which "counts" for facets, and should be "grouped" for presentation) is doctorid. When you offer users search over this index, you'd likely want to 1) group by distance (ie, < 0.1 miles, < 0.2 miles, etc., as a function query), but 2) also group by doctorid, ie cascaded grouping.
I suspect this would be easier to implement than it sounds: the per-group collector used by the 2nd pass grouping collector for key1's grouping just needs to be another grouping collector. Spookily, though, that collection would also have to be 2-pass, so it could get tricky since grouping is sort of recursing on itself.... once we have #4185, though, that should enable efficient single pass grouping by the identifier (doctorid).
---
Migrated from [LUCENE-3122](https://issues.apache.org/jira/browse/LUCENE-3122) by Michael McCandless (@mikemccand), updated May 09 2016
Contributor guide
Research direction
Start by reviewing the existing grouping collectors and the related SOLR-2526, #4170, and #4185 discussions. Clarify how first- and second-pass grouping currently work and whether collectors can be nested safely. Done means results are grouped first by key1 and then by key2 within each primary group, with the required two-pass behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100