vllm-project / vllm-project/aibrix

rayfleet scale down support ways to determine the scaling-down priority.

Open
#2,425 3 comments 0 reactions 1 assignee Claimed by @googs1025 View on GitHub
area/orchestration kind/bug
Dominant language
Go
Stars
5.1k
Forks
694
PR merge metrics
PR metrics pending

Description

### 🚀 Feature Description and Motivation

Currently, when scaling down a RayClusterReplicaSet, it directly removes the first N rayclusters in the list. It doesn't support scaling down specific replicas, which might result in deleting ready replicas while keeping unready ones, making the whole service unavailable. We need to support ways to determine the scaling-down priority of rayclusters, like sorting by deletion cost.

### Use Case

When reducing the number of replicas in rayfleet, the cluster list should be sorted in reverse order according to the deletion cost.

### Proposed Solution

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.