[BUG] Error while building the Cagra index with IVF PQ with error too many invalid or duplicated neighbor nodes
Nobody has claimed this yet.
- Dominant language
- Cuda
- Stars
- 854
- Forks
- 236
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 62
Description
Describe the bug
While building the cagra index using Faiss we are seeing this error
C++ exception RAFT failure at graph_core.cuh line=1417:
Could not generate an intermediate CAGRA graph because the initial kNN graph
contains too many invalid or duplicated neighbor nodes.
Some environment details:
libcuvs=25.08
cuda-nvcc 'cuda-version>=12.0,<=12.5'
cuda-toolkit=12.4.1
gxx_linux-64=12.4
Faiss commit: https://github.com/facebookresearch/faiss/tree/f9ccd582f9a9b8400428625d1ff3217ae83422b9
Job Failure details
| Parameter | Value |
|---|---|
| Dimension | 1024 |
| Total Docs (vectors) | 27,326 |
| Data Type | float32 |
| Space Type | L2 (Euclidean) |
CAGRA Index Configuration
| Parameter | Value | Derivation |
|---|---|---|
graph_degree |
32 | m * 2 → 16 × 2 |
intermediate_graph_degree |
64 | m * 4 → 16 × 4 |
graph_build_algo |
IVF_PQ | Default for float data |
n_lists |
165 | int(sqrt(doc_count)) → int(sqrt(27326)) |
pq_dim |
256 | dimension / 4 → 1024 / 4 |
refine_rate |
1.0 | Default (no refinement pass) |
store_dataset |
False | Default |
device |
0 | Default (single GPU) |
Error
RAFT failure at file=cpp/src/neighbors/detail/cagra/graph_core.cuh line=1417:
Could not generate an intermediate CAGRA graph because the initial kNN graph
contains too many invalid or duplicated neighbor nodes. This error can occur,
for example, if too many overflows occur during the norm computation between
the dataset vectors.
Warning Observed Before Failure
[warning] Self-included ratio is low: 4.41%. This can lead to poor recall.
Consider using a different configuration for the IVF-PQ index, increasing the
refinement rate, or using higher-precision data type for LUT/Internal Distance.
Call Stack
faiss.GpuIndexCagra::trainEx()
→ cuvs::neighbors::cagra::build()
→ cuvs::neighbors::cagra::detail::build()
→ cuvs::neighbors::cagra::detail::graph::optimize() ← FAILS HERE (line 1417)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at cpp/src/neighbors/detail/cagra/graph_core.cuh line 1417 and trace the graph optimize path shown in the call stack. Reproduce the failure with the reported 1,024-dimensional, 27,326-vector IVF_PQ configuration, then determine why invalid or duplicated neighbors are produced alongside the low self-included ratio. Done means the configuration no longer fails and the relevant behavior has regression coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning, search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100