zilliztech / zilliztech/VectorDBBench
🚀 Feature Request: Manual Search QPS Control
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 438
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 1
Description
Problem
Streaming tests only measure maximum QPS by running searches as fast as possible. No option exists to test performance under controlled, specific QPS loads.
Current vs Proposed
- Current: insert_rate controllable, search rate runs at maximum speed
- Proposed: Add optional search_qps parameter for controlled search rates
Use Cases
- SLA Testing: "Can DB maintain 95% accuracy at 1000 QPS?"
- Resource Planning: "What's latency at target 500 QPS?"
- Load Comparison: "DB A vs B performance at same controlled load"
- Bottleneck Analysis: "At what QPS does performance degrade?"
Expected Behavior
- When search_qps not specified → existing max QPS measurement (backward compatible)
- When search_qps specified → test performance at that specific rate
- Results show both target QPS and achieved metrics
Questions
- Does this feature align with project roadmap?
- Any preferred approach or naming conventions?
- Should this extend to other test types beyond streaming?
Next Steps
I'm planning to contribute this feature to VectorDBBench. Should I go ahead with the implementation?
Any guidance on preferred approach or design considerations would be appreciated before I start working on it.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the streaming benchmark's search loop and the existing insert_rate handling. Determine how an optional search_qps value can preserve maximum-speed behavior when omitted and report both target and achieved rates. Done means controlled-rate streaming tests work without changing existing defaults, with guidance from the unresolved design questions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100