vllm-project / vllm-project/aibrix
Enhance parallel download from HuggingFace
Open
area/model-loader
area/runtime
priority/important-soon
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 694
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 98
Description
### 🚀 Feature Description and Motivation
The current implementation of multi threading for downloading model from the HuggingFace is directly using
`max_workers` in `snapshot_download`.
> 1 thread = 1 file download
It doesn't do parallel downloading one file
### Use Case
_No response_
### Proposed Solution
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.