DrillSideways and early termination
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
As I was upgrading an application using Lucene 8.10 to Lucene 10.x, I've hit an unexpected situation with DrillSideways. In the past, when using DrillSideways with a collector that would early terminate, the whole collection would terminate and facet collection would also exit (with incomplete counts). It seems like now, DrillSideways#search code will wrap the hitCollector with a MultiCollector, and because of this, early termination in one collector doesn't translate into early termination in the other (the FacetCollector), so the collection will complete to the end. While I haven't test it, I expect FacetCollectorManager#search to behave similarly, since it's also wrapping the hitCollector in a MultiCollector.
Looking at the 8.x code, it seems like the behavior wasn't 100% consistent (i.e [when "drillDownDims" was empty](https://github.com/apache/lucene-solr/blob/faca81bff1a87ecbc6bc115d0cd79895fe12dd7e/lucene/facet/src/java/org/apache/lucene/facet/DrillSideways.java#L214) we would the and likely hit the same behavior) so this could have been a feature or a bug in 8.10 and before.
That said, I think this is a valid feature. I'm thinking we could have DrillSideways/FacetCollectorManager use a different implementation equivalent to MultiCollector that optionally exits collection whenever it sees the hitCollector throw a `CollectionTerminatedException`. Alternatively, if this is useful for features other than facet, we could change MultiCollector to optionally bubble up the `CollectionTerminatedException` instead of removing the particular collector that wants to terminate.
### Version and environment details
_No response_
Contributor guide
Research direction
Read DrillSideways#search and FacetCollectorManager#search, then compare their MultiCollector usage with the Lucene 8.x behavior described in the issue. Reproduce the interaction with an early-terminating hit collector and determine the expected CollectionTerminatedException behavior; done means the agreed behavior is covered for both paths without regressing facet counts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100