vllm-project / vllm-project/aibrix
Consider to integrate the LLM evaluation metrics to Autoscaling object
Open
area/autoscaling
kind/feature
priority/important-longterm
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 694
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 98
Description
### 🚀 Feature Description and Motivation
We already define a few autoscaling evaluation metrics like provision efficiency, SLO violations, resource usage etc.
If would be great for controller to evaluate it's autoscaling performance.
### Use Case
Help user understand how autoscaling are performed.
### Proposed Solution
Enable the evaluation or monitoring check and collect feedbacks in HPA object
Contributor guide
Assessment
This issue has not been assessed yet.