DrillSideways does not support intra-segment concurrency
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
With the introduction of intra-segment search concurrency (see #13542), `DrillSideways` is the only case that disables it in tests which can't support intra-segment slicing given its implementation requirements to go through all docs in a segment at once. This is asserted as well in its score method which checks that min and max are respectively `0` and `NO_MORE_DOCS`.
With intra-segment slicing randomly enabled in tests, we need to explicitly disable intra-segment concurrency in any test that relies on `DrillSideways`. That required adding an additional `newSearcher` method that takes in the supported concurrency: none, inter-segment or intra-segment.
This issue is to discuss a plan to support intra-segment concurrency in `DrillSideways`. Ideally, we'd be able to one day enable intra-segment slicing by default, once #13745 is addressed, but that would not make `IndexSearcher` usable in `DrillSideways` out of the box which is not a good user experience.
Contributor guide
Research direction
Start by reading the DrillSideways implementation and its score method, then review #13542 for intra-segment search concurrency and #13745 for the related limitation. Define a concrete design for supporting intra-segment slicing without requiring callers to disable concurrency, and document the expected behavior and tests needed to verify it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100