Fix IVF-PQ test threshold levels for cases with aggressive compression
Nobody has claimed this yet.
- Dominant language
- Cuda
- Stars
- 854
- Forks
- 236
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 62
Description
IVF-PQ speeds up by increasing the data compression level / reducing the precision during build (e.g. using pq_dim, pq_bits) and during search (adjusting the internal lookup table types) (see our blog post for more info).
These knobs reduce the achieved recall levels, and sometimes the penalty is stronger than that of the main search parameter n_probes.
The problem is that it's hard to know in advance what should be the expected recall level for various combinations of parameters, input data, and distance metrics. We attempt to do that in the tests, but there's always the compromise: either set the recall too high and risk occasional CI failures, or set it too low and risk missing a bug.
This issue is to track the progress/updates to the threshold logic.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in cpp/test/neighbors/ann_ivf_pq.cuh around lines 470-486, where recall thresholds are checked. Compare the threshold behavior across compression settings, search parameters, input data, and distance metrics. Done means the logic reflects expected recall tradeoffs while reliably detecting regressions without causing flaky CI failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- search, testing-qa
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100