vllm-project / vllm-project/aibrix
StreamLoader can persist models to disk in bypass
Open
area/model-loader
kind/enhancement
priority/important-soon
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 694
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 98
Description
### 🚀 Feature Description and Motivation
StreamLoader can persist models to disk in bypass. This way can avoid the network transmission overhead of loading the model from the current machine for the second time.
### Use Case
_No response_
### Proposed Solution
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.