AnswerDotAI / AnswerDotAI/fastkmeans

Feat: spherical / normalized centroids

Open
#5 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
104
Forks
8
PR merge metrics
No merged PRs in 30d

Description

Hello! In our own benchmark, `fastkmeans` achieved an **impressive 3x speedup** compared to the kmean of `faiss-gpu` that we used previously. Thanks for such a brilliant project!

In Faiss, when using `spherical=True` during Kmeans clustering, the centroids are normalized after each iteration. This will helps a lot when the original data metric is based on angular(Cosine or Dot) instead of L2.

https://github.com/facebookresearch/faiss/wiki/Faiss-building-blocks:-clustering,-PCA,-quantization#additional-options

I am curious if this feature could be applied in this amazing library.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the library's K-means entry point and inspect how centroids are updated between iterations. Compare the requested spherical behavior with Faiss's clustering option, then add coverage showing whether centroids are normalized after each iteration and document the resulting API behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.