cuvs-java: support pre-filter for all query types
Nobody has claimed this yet.
- Dominant language
- Cuda
- Stars
- 854
- Forks
- 236
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 62
Description
This issue has been filed to track adding support for a pre-filter to all query types, namely CagraQuery and HnswQuery.
Currently only BruteForceQuery supports a pre-filter. For example,
var queryBuilder = new BruteForceQuery.Builder()
.withPrefilter(new long[] { 0b1111L });
A pre-filter is important for the Lucene use case, since Lucene uses a pre-filter to, among other things, support filtering-out deleted docs while searching. (a deleted doc may still appear in the index, but is filtered out during search with a pre-filter). For now, we workaround this during search by oversampling the topK when searching an index that may have deleted docs or a user provided pre-filter. This is a rough heuristic and not ideal.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Java query types named in the issue, CagraQuery and HnswQuery, and compare their APIs with BruteForceQuery.Builder's existing withPrefilter example. The work is done when both query types support equivalent pre-filter input and the behavior covers Lucene's deleted-document and user-filter use cases without relying on oversampling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api, search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100