deepseek-ai / deepseek-ai/DeepSeek-VL
Adding support for quantization and fast inference server
Open
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 596
- PR merge metrics
- No merged PRs in 30d
Description
DeepSeek-VL 7B seems to be the awesome vision model that nobody is talking about. It is better than Llava 1.6 34B in VQA and OCR. It consistently return JSON when asked and just seems to have a great textual capability. Any recommendation on how to wrap DeepSeek-VL in a fast inference server and take for spin at scale ?
There are some issues opened here:
https://github.com/InternLM/lmdeploy/issues/1321
https://github.com/sgl-project/sglang/issues/297
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.