NVIDIA / NVIDIA/cuvs

[BUG] CAGRA build with IVF-PQ build algorithm

Open
#1,765 0 comments 0 reactions 1 assignee View on GitHub

@mfoerste4 is already working on this.

Since Feb 3, 2026.

bug
Dominant language
Cuda
Stars
854
Forks
236
Avg merge
3d 3h
Merged PRs (30d)
62

Description

When building a CAGRA index for the BIGANN-100M dataset using the internal IVF-PQ KNN build algorithm, the process encounters a significant quality issue. The system reports a critically low "Self-included ratio" of 0.39%, including that the initial KNN graph construction is failing to find even most basic neighbor relationships.

Error Message:
Building CAGRA index using IVF-PQ for KNN graph... [982986][22:55:47:687848][warning] Self-included ratio is low: 0.39 %. This can lead to poor recall. Consider using a different configuration for the IVF-PQ index, increasing the refinement rate, or using higher-precision data type for LUT/Internal Distance.
`

Enviornment & Reproduction:
Dataset: BigANN-100M (uint8_t base and query vectors).
Tested Architecture: Hopper (H100)

Current Parameters:

ivf_pq_params.build_params.n_lists = 16384;
ivf_pq_params.build_params.kmeans_n_iters = 25;
ivf_pq_params.build_params.kmeans_trainset_fraction = 0.1;
ivf_pq_params.build_params.conservative_memory_allocation = false;
ivf_pq_params.build_params.pq_bits = 8;
ivf_pq_params.build_params.pq_dim = 64;
ivf_pq_params.search_params.n_probes = 128; )
ivf_pq_params.search_params.lut_dtype = CUDA_R_32F;
ivf_pq_params.search_params.internal_distance_dtype = CUDA_R_32F;
ivf_pq_params.refinement_rate = 8.0;
// Assign to index params
index_params.intermediate_graph_degree = 64;
index_params.graph_degree = 48;
index_params.graph_build_params = ivf_pq_params;

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.