facebookresearch / facebookresearch/faiss

IVFPQfs slower than benchmark with version:1.7.1 on sift1M

Open
#2,199 0 comments 0 reactions 0 assignees View on GitHub
install Performance unconfirmed-bug
Dominant language
C++
Stars
40.9k
Forks
4.5k
PR merge metrics
No merged PRs in 30d

Description

# Summary

IVF1024,PQ64x4fs is very fast exactly, but almost slower 2x ~ 3x times than benchmarks both python (installed fromconda) and C++ (Built from source by myself). Is there something wrong with my config? or just because of the different machines.
almost slower than benchmarks , sift1M almost slower
# Platform

OS:
ubuntu 16.04
Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz
Faiss version:
4dcb0a5eac8f84d26570a0fb6f3e357bc2f1151c
Installed from:

Faiss compilation options:
using MKL and FAISS_OPT_LEVEL=avx2
Running on:
- [x] CPU

- [ ] GPU

Interface:
- [x] C++
- [x] Python

# Reproduction instructions

1. Experiment1

compare IVFPQfs with benchmarks(IVF1024-enconding) by conda.
python bench_all_ivf.py --indexkey=IVF1024,PQ64x4fs --db=sift1M --searchthreads=32 (in benchs/bench_all_ivf)

result:
parameters R@1 R@10 R@100 Qps(q/s) nb distances #runs
nprobe=1 0.3035 0.4479 0.4575 651957.18266 11657170 196
nprobe=2 0.3859 0.6041 0.6209 533307.26751 23321409 160
nprobe=4 0.4474 0.7418 0.7683 337547.01956 46377475 102
nprobe=8 0.4901 0.8437 0.8821 260789.07285 91350959 79
nprobe=16 0.5114 0.9020 0.9510 205776.14671 178568844 62
nprobe=32 0.5227 0.9300 0.9858 124097.45755 347706719 38

**slower than benchmarks in IVF1024-encoding (IVF1024,PQ64x4fs 32bytes),** is there some different config?

2. Experiment2

repeat the above experiment by C++ mode,
cmake -DFAISS_ENABLE_GPU=OFF -DBLA_VENDOR=Intel10_64_dyn -DFAISS_OPT_LEVEL=avx2 -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=/opt/intel/mkl .

my code:

// init
size_t dim = 128;
size_t ncentroids = 1024;
size_t M = 64;
size_t nbits = 4;
index_flat_.reset(new faiss::IndexFlatL2(dim));
std::shared_ptr index_;
index_.reset(new faiss::IndexIVFPQFastScan(index_flat_.get(), dim, ncentroids, M, nbits, faiss::METRIC_L2));
// train 100000 * 128D
index_->train(n, p_feat);

// add 1000000 * 128D
index_->add(n, p_feat);

// search 10000 * 128D
for (nprobe in {1 2 4 8 16 32 64}){
index->nprobe = nprobe;
index_->search(n, p_feat, k, scores, ids);
}

result:
parameters R@1 R@10 R@100 Qps(q/s)
nprobe=1 0.3040 0.4405 0.4576 344827
nprobe=2 0.3820 0.5819 0.6130 285714
nprobe=4 0.4423 0.7024 0.7527 294117
nprobe=8 0.4832 0.7853 0.8492 227272
nprobe=16 0.5016 0.8286 0.9067 166666
nprobe=32 0.5085 0.8525 0.9332 94339

**C++ mode slower 2x than python mode(Experiment1) and slower 3x~4x than benchmarks.**

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.