vllm-project / vllm-project/aibrix
Define and validate Recommended AIBrix configurations for Production Use
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 697
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 104
Description
### 🚀 Feature Description and Motivation
Currently, many controller configuration values (e.g., resources, concurrency limits, resync periods, queue lengths) are set based on heuristics or "magic numbers" without empirical justification. This can lead to instability, performance bottlenecks, or memory issues at scale.
To support production-grade deployment, we need to identify recommended configuration ranges through systematic testing and monitoring.
### Goals
- Establish recommended settings for different size of clusters. A documented guide of production-safe configuration profiles (small, medium, large clusters).
- performance and scale testing.
- stability and memory validation through long running services.
### Use Case
for production user
### Proposed Solution
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.