deepseek-ai / deepseek-ai/DeepSeek-Coder
用vllm加速推理框架 推理速度还是很慢
Open
- Dominant language
- Python
- Stars
- 24.3k
- Forks
- 2.9k
- PR merge metrics
- No merged PRs in 30d
Description
本地部署的deepseek-coder-33b，两块RTX A6000 48G的显卡
python -m vllm.entrypoints.openai.api_server --model /home/superadmin/coder33b --trust-remote-code --tensor-parallel-size=2 --served-model-name=deepseek-coder
启动后调用/v1/completions接口获取推理结果，但是效率极低，平均15tokens/s，如图
有人遇到过吗，如何解决有大佬指教吗
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.