deepseek-ai / deepseek-ai/DeepSeek-V2
模型部署困惑
Open
- Dominant language
- No language data
- Stars
- 5k
- Forks
- 550
- PR merge metrics
- No merged PRs in 30d
Description
在A800 8卡机上按照Chat Completion的示例代码加载模型
设置
max_memory = {i: "75GB" for i in range(8)}
device_map="sequential"
启动后显存占用集中在0卡~6卡，7卡被完全放空，然后测试的上下文稍长一点，就报显存无法分配了
这是什么原因呢，有人遇到过吗
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.