[QST] ARM outperforms x86 on hnswlib by up to 2x
Nobody has claimed this yet.
- Dominant language
- Cuda
- Stars
- 854
- Forks
- 236
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 62
Description
Ran cuvs-bench on a 550k dataset (BeIR/trec-news-generated-queries on Hugging Face with duplicates removed) on AWS i8g and g6 (NVDIA A10G) instances with 32 vCPU (8xlarge) and 64 vCPU (16xlarge) using hnswlib. The data was embedded using google/embeddinggemma-300m using the default 768 dimension, and casted to fp32 for compatibility with CAGRA. Results are shown below for the parameter sweep using throughput mode. It is observed that a 32 vCPU ARM system performed better than the 64 vCPU x86 system by up to 50%. Is this result expected?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the cuvs-bench throughput results described for hnswlib, the 550k BeIR dataset, and the AWS i8g and g6 configurations. Reproduce the comparison between the 32-vCPU ARM system and 64-vCPU x86 system, then determine whether the observed difference is expected and record the relevant explanation or benchmark findings.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- performance, search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100