deepseek-ai / deepseek-ai/DeepSeek-V2
why i use vllm inference deepseek v2 ,speed is low
Open
- Dominant language
- No language data
- Stars
- 5k
- Forks
- 550
- PR merge metrics
- No merged PRs in 30d
Description
i use vllm to inference deepspeed, use flask to deploy model. When the problem enters the model, it always gets stuck for a long time in the processd prompt step，the code i use is your example code
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.