Is possible to run this model with ROCm?
未關閉
- 主要語言
- Python
- 星號
- 19.5k
- 分支
- 1.6k
- PR 合併指標
- 30 天內沒有已合併 PR
描述
I have a 7900 XTX, a ROCm enabled card with 24GB of VRAM
I tried install vLLM but it just got OOM maybe because it was no flash-attention support.
tried ollama with GGUF version, but it just returns me empty response.
貢獻指南
評估
這個 Issue 還沒有評估資料。