NVIDIA / NVIDIA/cuvs

Lucene: using cagraHnswLayers > 1 and IVF-PQ with cagraIntermediateGraphDegree >= 128 results in extra slow performance

Open
#2,471 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Lucene
Dominant language
Cuda
Stars
854
Forks
236
Avg merge
3d 3h
Merged PRs (30d)
62

Description

Reproduction steps:

  1. run SearchScale benchmark script on the latest 26.08.0 main branches of cuvs and cuvs-lucene
  2. Need to use a dataset with at least 5M vectors in order to trigger IVF-PQ
  3. Need to use a dataset with a distribution that can handle setting cagraHnswLayers above 1 (e.g. 3)
  4. run exactly two tests where for one of them you set cagraIntermediateGraphDegree=126 and another one where you set cagraIntermediateGraphDegree=128
  5. Observe the results

Results:
cagraGraphDegree = 66, cagraIntermediateGraphDegree = 126:
Total Index Build Time: 580 seconds

cagraGraphDegree = 66, cagraIntermediateGraphDegree = 128:
Total Index Build Time: 1058 seconds

Note:
Reproducing this may require the PR to enable out-of-core index-building https://github.com/rapidsai/cuvs-lucene/pull/141
Alternatively, if your dataset can fit on your GPU, then this is reproducible on multiple GPUs (e.g. I have reproduced it on both an A10-G and an L40S)

Please see attachment for detailed metrics:

metrics_ySiC8I_GPU__10M-1536d__1-seg_3-hnsw-layers__corner-case-128.csv

Note: I do not see this to be reproducible when using NN_DESCENT or when setting cagraHnswLayers=1. This is an issue because cuVS limits us to cagraIntermediateGraphDegree <= 512, and sometimes we may need to set cagraHnswLayers > 1 to achieve sought recall. This is 100% reproducible

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the SearchScale benchmark script against the latest cuvs and cuvs-lucene branches, using a 5M+ vector dataset and cagraHnswLayers=3. Compare cagraIntermediateGraphDegree values 126 and 128, using PR 141 for out-of-core setup if needed; done means identifying and correcting the reproducible build-time regression while preserving the reported configurations.

Written by the indexing model from the issue text.

Assessment

Domain
performance, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.