[BUG] Single gpu K-means peak memory footprint is 2x the dataset size.
@HowardHuang1 is already working on this.
Since Feb 7, 2026.
- Dominant language
- Cuda
- Stars
- 854
- Forks
- 236
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 62
Description
Describe the bug
Ran kmeans with variable dataset size and monitored gpu memory at 1 sec interval via pynvml. Suspecting that there is a dataset copy that is leading to a near 2x peak memory overhead. This also limits the largest shard that can be clustered per gpu hence impacting the search accuracy of the index.
Steps/Code to reproduce bug
GITHUB
Installation: Follow the instructions in the Github README.md
Execution: Run this notebook for running clustering and monitoring.
Dataset: Bao to share 10M, 1024D, FP32; MIRACL embeddings (via slack / google drive).
Expected behavior
peak gpu memory overhead = 1-1.2x dataset_size
Environment details (please complete the following information):
- Environment location: On-prem
- Method of cuVS install: conda
- Machine Specs: 1xA100 (80 GB), 128 vCPU (2x AMD EPYC 7532 @ 2.4GHz)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.