NVIDIA / NVIDIA/cuvs

[FEA] CAGRA attach_dataset_on_build Python API support

Open
#806 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature request
Dominant language
Cuda
Stars
854
Forks
236
Avg merge
3d 3h
Merged PRs (30d)
62

Description

Is your feature request related to a problem? Please describe.
The CAGRA C++ API has support for the attach_dataset_on_build parameter: https://docs.rapids.ai/api/cuvs/stable/cpp_api/neighbors_cagra/#_CPPv423attach_dataset_on_build. However, the Python API does not: https://docs.rapids.ai/api/cuvs/stable/python_api/neighbors_cagra/#index-build-parameters. When I try to create a cagra.IndexParams object in Python like so:

    index_params = cagra.IndexParams(intermediate_graph_degree=64,graph_degree=32,build_algo='ivf_pq', metric="sqeuclidean", attach_dataset_on_build=False)

I get the following error:

  File "cagra.pyx", line 165, in cuvs.neighbors.cagra.cagra.IndexParams.__init__
TypeError: __init__() got an unexpected keyword argument 'attach_dataset_on_build'

Describe the solution you'd like

Support the attach_dataset_on_build parameter in cuvs cagra Python API

Describe alternatives you've considered
N/A

Additional context
I would like to build a CAGRA index without loading the entire dataset into GPU memory, using Python. It looks like it's only possible to do this with the C++ API, as of now.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the Python CAGRA entry point shown in the traceback, cagra.pyx at IndexParams.init around line 165, and compare its accepted parameters with the C++ API and Python API documentation. Done means Python accepts attach_dataset_on_build when constructing cagra.IndexParams and supports building without loading the entire dataset into GPU memory.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.