NVIDIA / NVIDIA/cuvs

[FEA] Improved support for building graph-based indexes on lower dimensional vectors

Open
#873 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature request
Dominant language
Cuda
Stars
854
Forks
236
Avg merge
3d 3h
Merged PRs (30d)
62

Description

While CAGRA, DIskANN and other index types tend to build indexes much faster for higher dimensional vectors on the GPU (e.g 10x+ speedup), we are finding that compressed, heavily quantized, and lower dimensional vectors (eg. 64 dims) are not as competitive and are yielding build speedups that are just not acceptable in practice (e.g. 4x or less) to convince users to invest in the complexity of GPUs.

The working theory is that the lower dimensions aren't able to fully saturate the memory bandwidth and cause low GPU utililization in general. This is also in part because the each vector tends to use random io, and so when each vector is not large enough to saturate the coalescing bandwidth, we leave a lot of perf on the table.

There's an assumption here that we can update the grids in the CUDA kernels to do more vectorized loads, or chunking within each warp so that we can improve the perf. We need to investigate the possiblities here because, while LLMs and RAG have typically driven the vector widths up, we are beginning to see a general shift towards compression, quantization, and smaller dims as a means to 1) scale, and 2) save on disk/memory costs.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files or tests are named. Start by profiling CAGRA, DiskANN, and other graph-index build paths for lower-dimensional vectors, then inspect the CUDA kernels and their grid, load, and warp behavior. Done means demonstrating improved build performance for compressed or roughly 64-dimensional vectors, but the issue does not define a target speedup.

Written by the indexing model from the issue text.

Assessment

Domain
performance, search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.