[BUG] ivf_rabitq needs seralize & deserilize steps before the index is searchable
Nobody has claimed this yet.
- Dominant language
- Cuda
- Stars
- 854
- Forks
- 236
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 62
Description
Describe the bug
If I run ivf_rabitq::build() followed by ivf_rabitq::search(), it will not return correct neighbors. In contrast, other indices work correctly: an index reutrned by build() is searchable (assuming it fits memory).
To get correct results from the ivf_rabitq index we need to save to file and load again. This step is included in cuvs-bench as well as the unit tests.
Steps/Code to reproduce bug
In the following example if save_load is true, then recall would be correct, otherwise 0.
cuvs::neighbors::ivf_rabitq::index_params index_params;
index_params.n_lists = n_lists;
index_params.bits_per_dim = bits_per_dim;
auto index = cuvs::neighbors::ivf_rabitq::build(
res, index_params, raft::make_const_mdspan(d_dataset.view()));
std::string save_path = "ivf_rabitq_minimal.ibin";
if (save_load) {
cuvs::neighbors::ivf_rabitq::serialize(res, save_path, index);
cuvs::neighbors::ivf_rabitq::deserialize(res, save_path, &index);
}
cuvs::neighbors::ivf_rabitq::search_params search_params;
search_params.n_probes = n_probes;
cuvs::neighbors::ivf_rabitq::search(res,
search_params,
index,
raft::make_const_mdspan(d_queries.view()),
d_neighbors.view(),
d_distances.view());
Expected behavior
It is expected that we can search right after build, without saving/loading the dataset.
Additional context
Full example code to illustrate the bug
https://gist.github.com/tfeher/77f9e956f4bc66747fb5be0996961c54
Usage
./RABITQ_MINIMAL 10000 128 64 8 8 rabitq 0
./RABITQ_MINIMAL 10000 128 64 8 8 rabitq 1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the cuvs::neighbors::ivf_rabitq build, serialize, deserialize, and search entry points, then compare the save/load path used by cuvs-bench and the unit tests. Reproduce the minimal example with save_load disabled and verify that searching immediately after build returns the same correct neighbors as the serialized and deserialized index.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- search
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100