vllm-project / vllm-project/aibrix

Delegate model adapter credential to runtime to download model adapters

Open
#1,826 1 comment 0 reactions 1 assignee Claimed by @scarlet25151 View on GitHub
Dominant language
Go
Stars
5.1k
Forks
697
Avg merge
1d 19h
Merged PRs (30d)
104

Description

### 🚀 Feature Description and Motivation

Model Adapter could come from different object storage. We can not assume the base model can only consume models from same object store.

In order to achieve the flexible setting, we should consider to assume the roles and allocate short term tokens to runtime to download the model

part of #1818

### Use Case

for model adapter downloading

### Proposed Solution

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.