NVIDIA / NVIDIA/cuvs

Fix IVF-PQ test threshold levels for cases with aggressive compression

Open
#581 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Cuda
Stars
854
Forks
236
Avg merge
3d 3h
Merged PRs (30d)
62

Description

IVF-PQ speeds up by increasing the data compression level / reducing the precision during build (e.g. using pq_dim, pq_bits) and during search (adjusting the internal lookup table types) (see our blog post for more info).

These knobs reduce the achieved recall levels, and sometimes the penalty is stronger than that of the main search parameter n_probes.

The problem is that it's hard to know in advance what should be the expected recall level for various combinations of parameters, input data, and distance metrics. We attempt to do that in the tests, but there's always the compromise: either set the recall too high and risk occasional CI failures, or set it too low and risk missing a bug.

This issue is to track the progress/updates to the threshold logic.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in cpp/test/neighbors/ann_ivf_pq.cuh around lines 470-486, where recall thresholds are checked. Compare the threshold behavior across compression settings, search parameters, input data, and distance metrics. Done means the logic reflects expected recall tradeoffs while reliably detecting regressions without causing flaky CI failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
search, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.