[BUG] ANN_BENCH: incorrect output index type in multi-gpu CAGRA
Nobody has claimed this yet.
- Dominant language
- Cuda
- Stars
- 854
- Forks
- 236
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 62
Description
All multi-gpu implementations in ANN_BENCH ignore the benchmark index type algo_base::index_type, which is int64_t:
- https://github.com/rapidsai/cuvs/blob/73e4fbcb981a67048a7f63b39416a87cc878c1ba/cpp/bench/ann/src/cuvs/cuvs_mg_cagra_wrapper.h#L161-L162
- https://github.com/rapidsai/cuvs/blob/73e4fbcb981a67048a7f63b39416a87cc878c1ba/cpp/bench/ann/src/cuvs/cuvs_mg_ivf_flat_wrapper.h#L134-L135
- https://github.com/rapidsai/cuvs/blob/73e4fbcb981a67048a7f63b39416a87cc878c1ba/cpp/bench/ann/src/cuvs/cuvs_mg_ivf_pq_wrapper.h#L131-L132
IVF-Flat and IVF-PQ index types coincide with the benchmark index type and thus show correct results.
CAGRA index type is always instantiated as uint32_t. As a result, int64_t array is casted to int32_t, and the benchmark interprets the algorithm output incorrectly.
We either need to map the outputs correctly as it's done in IVF-PQ benchmark wrapper (which adds extra overhead if the types do not match and puts the index at disadvantage in the benchmarks) or adjust the implementation of multi-gpu indexes to decouple index-index type from the search output index type as it was done for CAGRA in https://github.com/rapidsai/cuvs/pull/769 .
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the multi-GPU wrappers at cpp/bench/ann/src/cuvs/cuvs_mg_cagra_wrapper.h:161-162, cuvs_mg_ivf_flat_wrapper.h:134-135, and cuvs_mg_ivf_pq_wrapper.h:131-132, then compare their output handling with cuvs_ivf_pq_wrapper.h:181-222 and cuvs_cagra_wrapper.h:368-369. Decide whether to map output types or decouple the index and search output types, then run ANN_BENCH to confirm multi-GPU CAGRA results use the benchmark's int64_t index type.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100