deepseek-ai / deepseek-ai/DeepSeek-VL
4-bit quant?
Open
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 596
- PR merge metrics
- No merged PRs in 30d
Description
Hi! Thank you for releasing this multimodal model. First test are impressive. Even 1.3B is good for its size.
It is just that 7b version in full precision is still taxing on personal HW we have at home.
Would it be possible to quantize it to int4 like Qwen did with their Qwen-VL-Chat-Int4?
I think it would be best if you could do it and put it in your HF repo so community can use it.
If not, maybe you could give us some guidelines how to do it.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.