vllm-project / vllm-project/aibrix

Consider to integrate the LLM evaluation metrics to Autoscaling object

Open
#65 1 comment 0 reactions 1 assignee Claimed by @kr11 View on GitHub
area/autoscaling kind/feature priority/important-longterm
Dominant language
Go
Stars
5.1k
Forks
694
Avg merge
1d 20h
Merged PRs (30d)
98

Description

### 🚀 Feature Description and Motivation

We already define a few autoscaling evaluation metrics like provision efficiency, SLO violations, resource usage etc.

If would be great for controller to evaluate it's autoscaling performance.

### Use Case

Help user understand how autoscaling are performed.

### Proposed Solution

Enable the evaluation or monitoring check and collect feedbacks in HPA object

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.