facebookresearch / facebookresearch/faiss
IVFPQfs slower than benchmark with version:1.7.1 on sift1M
- Dominant language
- C++
- Stars
- 40.9k
- Forks
- 4.5k
- PR merge metrics
- No merged PRs in 30d
Description
# Summary
IVF1024,PQ64x4fs is very fast exactly, but almost slower 2x ~ 3x times than benchmarks both python (installed fromconda) and C++ (Built from source by myself). Is there something wrong with my config? or just because of the different machines.
almost slower than benchmarks , sift1M almost slower
# Platform
OS:
ubuntu 16.04
Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz
Faiss version:
4dcb0a5eac8f84d26570a0fb6f3e357bc2f1151c
Installed from:
Faiss compilation options:
using MKL and FAISS_OPT_LEVEL=avx2
Running on:
- [x] CPU
- [ ] GPU
Interface:
- [x] C++
- [x] Python
# Reproduction instructions
1. Experiment1
compare IVFPQfs with benchmarks(IVF1024-enconding) by conda.
python bench_all_ivf.py --indexkey=IVF1024,PQ64x4fs --db=sift1M --searchthreads=32 (in benchs/bench_all_ivf)
result:
parameters R@1 R@10 R@100 Qps(q/s) nb distances #runs
nprobe=1 0.3035 0.4479 0.4575 651957.18266 11657170 196
nprobe=2 0.3859 0.6041 0.6209 533307.26751 23321409 160
nprobe=4 0.4474 0.7418 0.7683 337547.01956 46377475 102
nprobe=8 0.4901 0.8437 0.8821 260789.07285 91350959 79
nprobe=16 0.5114 0.9020 0.9510 205776.14671 178568844 62
nprobe=32 0.5227 0.9300 0.9858 124097.45755 347706719 38
**slower than benchmarks in IVF1024-encoding (IVF1024,PQ64x4fs 32bytes),** is there some different config?
2. Experiment2
repeat the above experiment by C++ mode,
cmake -DFAISS_ENABLE_GPU=OFF -DBLA_VENDOR=Intel10_64_dyn -DFAISS_OPT_LEVEL=avx2 -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=/opt/intel/mkl .
my code:
// init
size_t dim = 128;
size_t ncentroids = 1024;
size_t M = 64;
size_t nbits = 4;
index_flat_.reset(new faiss::IndexFlatL2(dim));
std::shared_ptr index_;
index_.reset(new faiss::IndexIVFPQFastScan(index_flat_.get(), dim, ncentroids, M, nbits, faiss::METRIC_L2));
// train 100000 * 128D
index_->train(n, p_feat);
// add 1000000 * 128D
index_->add(n, p_feat);
// search 10000 * 128D
for (nprobe in {1 2 4 8 16 32 64}){
index->nprobe = nprobe;
index_->search(n, p_feat, k, scores, ids);
}
result:
parameters R@1 R@10 R@100 Qps(q/s)
nprobe=1 0.3040 0.4405 0.4576 344827
nprobe=2 0.3820 0.5819 0.6130 285714
nprobe=4 0.4423 0.7024 0.7527 294117
nprobe=8 0.4832 0.7853 0.8492 227272
nprobe=16 0.5016 0.8286 0.9067 166666
nprobe=32 0.5085 0.8525 0.9332 94339
**C++ mode slower 2x than python mode(Experiment1) and slower 3x~4x than benchmarks.**
Contributor guide
Assessment
This issue has not been assessed yet.