[BUG] IVF-PQ support of IP distance assumes vectors have been normalized
Open
@HowardHuang1 is already working on this.
Since Mar 10, 2026.
bug
- Dominant language
- Cuda
- Stars
- 854
- Forks
- 236
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 62
Description
IVFPQ w/ IP distance assumes vectors have been normalized, but not all users would expect or think to do this. Essentially, kmeans works with L2 distance or with IP and l2 normalized vectors. Currently if IP is used with vectors that aren't normalized then it runs into degenerate cases where most vectors are placed in a single cluster, and that causes issues for downstream algorithms (like the IVF portions).
One way to deal with this is to do normalization in-place or to normalize the input dataset then return it to its original state after the clustering is done.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.