CAGRA -> HNSWLIB Poor Recall on Wiki Dataset

Dataset

97M wiki embeddings sourced from https://huggingface.co/datasets/Cohere/wikipedia-2023-11-embed-multilingual-v3-int8-binary
Query set: 10000 randomly sampled queries from this set (these were removed from the base training set)

Dataset shape

97M * 1024 int8 embeddings

Issue

With cuvs-bench, HNSWLIB recall goes to as high as 99% whereas CAGRA -> HNSWLIB the recall only gets up to 91%

cuvs-bench Index Params:

graph_build_algo: ["IVF_PQ"]
ivf_pq_build_kmeans_trainset_ratio: [10]
ivf_pq_build_pq_dim: [128]
ivf_pq_build_pq_bits: [8]
ivf_pq_search_n_probes: [2]
ivf_pq_search_refinement_rate: [1]
graph_degree: [32, 64, 96, 128]
intermediate_graph_degree: [32, 64, 96, 128]
hierarchy: ["cpu"]
ef_construction: [64, 128, 256, 512]

Start by reproducing the reported recall gap with cuvs-bench, the Wiki embeddings dataset, and the listed IVF_PQ, graph-degree, hierarchy, and ef-construction settings. Compare CAGRA -> HNSWLIB results with direct HNSWLIB results; done means identifying and correcting the cause of the recall difference or documenting a reproducible limitation.