facebookresearch / facebookresearch/faiss

memory leak when index.search

Open
#2,170 6 comments 0 reactions 0 assignees View on GitHub
cant-repro Implementation unconfirmed-bug
Dominant language
C++
Stars
40.9k
Forks
4.5k
PR merge metrics
No merged PRs in 30d

Description

```
import faiss
from facebank import *
import pickle
import time
import gc
#index = faiss.index_factory(512, "IVF16384, Flat")
index = faiss.read_index('log/features_exp2/feature_00550220.index')
index_ivf = faiss.extract_index_ivf(index)
index_ivf.nprobe = 50

with open('log/features_exp2/records.add.00000000_00023729.pkl','rb') as f:
features = pickle.loads(f.read())[0].data #a np.array([1,512], np.float32)
for i in range(100000):
rst = index.search(features,1)
if i % 100 ==0:
print(faiss.get_mem_usage_kb())

#output
1244664
1248024
1249692
1252228
1253488
1256024
1257284
1259820
1261080
1263616
1265988
1267676
1270200
1271472
1274008
1275268
...
```
Hello, Is that normal the memory keeps increasing when search? Is it a memory leak?
faiss==1.7.1.post3

Contributor guide

Open the contributing guide

Research direction

Reproduce the supplied Python loop with faiss==1.7.1.post3, starting from faiss.read_index, index.search, and faiss.get_mem_usage_kb. Check whether memory growth persists and whether it is allocator behavior or retained search state. Done means documenting a confirmed diagnosis and, if a leak is confirmed, providing a regression test and fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.