Add support for subsets of searchables inside a MultiSearcher/ParallelMultiSearcher instance's methods at runtime [LUCENE-2447]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
Here's the situation: We have a site with a fair few amount of indexes that we're using MultiSearcher/ParallelMultiSearcher for, but the users can select an arbitrary permutation of indexes to search. For example (contrived, but illustratory): the site has indexes numbered 1 - 10; user A wants to search in all 10; user B wants to search indexes 1, 2 and 3, user C wants to search even-numbered indexes. From Lucene 3.0.1, the only way to do this is to continually instantiate a new MultiSearcher based on every permutation of indexes that a user wants, which is not ideal at all.
What I've done is add a new parameter to all methods in MultiSearcher that use the searchables array (docFreq, search, rewrite and createDocFrequencyMap), a Set<Searchable> which is checked for isEmpty() and contains() for every iteration over the searchables[]. The actual logic has been moved into these methods and the old methods have become overloads that pass a Collections.emptySet() into those methods, so I do not expect there to be a very noticeable performance impact as a result of this modification, if it's measurable at all.
I didn't modify the test for MultiSearcher very much, just enough to illustrate the that subsetting of the search results works, since no other logic has changed. If I need to do more for the testing, let me know and I'll do it.
I've attached the patches for MultiSearcher.java, ParallelMultiSearcher.java and TestMultiSearcher.java.
---
Migrated from [LUCENE-2447](https://issues.apache.org/jira/browse/LUCENE-2447) by Edward Drapkin, updated May 06 2010
Environment:
```
Irrelevant
```
Attachments: [LUCENE-2447.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-2447/LUCENE-2447.patch), [LUCENE-2447-predicate.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-2447/LUCENE-2447-predicate.patch)
Contributor guide
Research direction
Start with MultiSearcher.java and ParallelMultiSearcher.java, then review TestMultiSearcher.java and the attached LUCENE-2447 patches. Run the existing MultiSearcher tests first and compare the proposed runtime subset behavior with the current overloads. Done means supported methods can search a selected subset of searchables while existing behavior remains covered by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100