NVIDIA / NVIDIA/cuvs

[BUG] ANN_BENCH: incorrect output index type in multi-gpu CAGRA

Open
#894 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Cuda
Stars
854
Forks
236
Avg merge
3d 3h
Merged PRs (30d)
62

Description

All multi-gpu implementations in ANN_BENCH ignore the benchmark index type algo_base::index_type, which is int64_t:

IVF-Flat and IVF-PQ index types coincide with the benchmark index type and thus show correct results.
CAGRA index type is always instantiated as uint32_t. As a result, int64_t array is casted to int32_t, and the benchmark interprets the algorithm output incorrectly.

We either need to map the outputs correctly as it's done in IVF-PQ benchmark wrapper (which adds extra overhead if the types do not match and puts the index at disadvantage in the benchmarks) or adjust the implementation of multi-gpu indexes to decouple index-index type from the search output index type as it was done for CAGRA in https://github.com/rapidsai/cuvs/pull/769 .

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the multi-GPU wrappers at cpp/bench/ann/src/cuvs/cuvs_mg_cagra_wrapper.h:161-162, cuvs_mg_ivf_flat_wrapper.h:134-135, and cuvs_mg_ivf_pq_wrapper.h:131-132, then compare their output handling with cuvs_ivf_pq_wrapper.h:181-222 and cuvs_cagra_wrapper.h:368-369. Decide whether to map output types or decouple the index and search output types, then run ANN_BENCH to confirm multi-GPU CAGRA results use the benchmark's int64_t index type.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.