vllm-project / vllm-project/aibrix
Delegate model adapter credential to runtime to download model adapters
Open
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 697
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 104
Description
### 🚀 Feature Description and Motivation
Model Adapter could come from different object storage. We can not assume the base model can only consume models from same object store.
In order to achieve the flexible setting, we should consider to assume the roles and allocate short term tokens to runtime to download the model
part of #1818
### Use Case
for model adapter downloading
### Proposed Solution
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.