[BUG] `raft-ann-bench` fails to benchmark the build time of `faiss_cpu_flat`.

Open
#2,148 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
docker, python
Domain
performance, testing-qa

Research direction

Start by running the provided Docker reproduction for python -m raft-ann-bench.run with faiss_cpu_flat, then inspect the FAISS_CPU_FLAT_ANN_BENCH invocation and its output paths. Done means the benchmark no longer segfaults and creates datasets/glove-100-inner/result/build/faiss_cpu_flat,base.json alongside the saved index.

Written by the indexing model from the issue text.

Description

bug Vector Search

Describe the bug

raft-ann-bench fails to benchmark the build time of faiss_cpu_flat. The index seems to build successfully, as the index file is saved, but the result file is not created.

Steps/Code to reproduce bug

docker run --rm -u $(id -u) --gpus all --entrypoint /bin/bash --workdir /data/benchmarks -v .:/data/benchmarks rapidsai/raft-ann-bench:24.02a-cuda11.8-py3.10 -c 'eval "$(conda shell.bash hook)"
python -m raft-ann-bench.run --build --dataset glove-100-inner --algorithms faiss_cpu_flat --force'

Expected behavior

  • An index file is built and saved as ./datasets/glove-100-inner/index/faiss_cpu_flat
  • Missing: A result file is produced ./datasets/glove-100-inner/result/build/faiss_cpu_flat,base.json

Additional context
Error message

Error occurred running benchmark: Command '['/opt/conda/bin/ann/FAISS_CPU_FLAT_ANN_BENCH', '--build', '--data_prefix=/data/benchmarks/datasets/', '--benchmark_out_format=json', '--benchmark_counters_tabular=true', '--benchmark_out=/data/benchmarks/datasets/glove-100-inner/result/build/faiss_cpu_flat,base.json.lock', '--raft_log_level=3', '--force', 'glove-100-inner_faiss_cpu_flat,base,k10,bs10000_a3f4f69a-c183-11ee-ad15-0242ac110003.json']' died with <Signals.SIGSEGV: 11>.
Dominant language
Cuda
Stars
1k
Forks
251
Avg merge
1d 8h
Merged PRs (30d)
31

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA/raft

All issues in NVIDIA/raft

Similar issues

More Performance issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.