NVIDIA / NVIDIA/cuvs

cuvs-java: support pre-filter for all query types

Open
#696 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Cuda
Stars
854
Forks
236
Avg merge
3d 3h
Merged PRs (30d)
62

Description

This issue has been filed to track adding support for a pre-filter to all query types, namely CagraQuery and HnswQuery.

Currently only BruteForceQuery supports a pre-filter. For example,

var queryBuilder = new BruteForceQuery.Builder()
   .withPrefilter(new long[] { 0b1111L });

A pre-filter is important for the Lucene use case, since Lucene uses a pre-filter to, among other things, support filtering-out deleted docs while searching. (a deleted doc may still appear in the index, but is filtered out during search with a pre-filter). For now, we workaround this during search by oversampling the topK when searching an index that may have deleted docs or a user provided pre-filter. This is a rough heuristic and not ideal.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Java query types named in the issue, CagraQuery and HnswQuery, and compare their APIs with BruteForceQuery.Builder's existing withPrefilter example. The work is done when both query types support equivalent pre-filter input and the behavior covers Lucene's deleted-document and user-filter use cases without relying on oversampling.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, search
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.