deepseek-ai / deepseek-ai/DeepSeek-VL2
VL2 Small produces wrong response for an input image
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 1.8k
- PR merge metrics
- No merged PRs in 30d
Description
model_path = "deepseek-ai/deepseek-vl2-small"
vl_chat_processor: DeepseekVLV2Processor = DeepseekVLV2Processor.from_pretrained(model_path)
tokenizer = vl_chat_processor.tokenizer
#model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map="auto")
quantization_config = QuantoConfig(weights="int8")
vl_gpt: DeepseekVLV2ForCausalLM = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype="auto", device_map="cuda:0", trust_remote_code=True, quantization_config=quantization_config)
I update the image and question in the Conversation:
(image below)

I then ask the question: Describe the image in detail.
The response I get back is as follows after 2 successive tries:
<|Assistant|>: The image shows a person standing on a beach, with their back to the camera. The person is wearing a white shirt and blue shorts, and they have a backpack on their shoulders. The beach is sandy and there are no other people visible in the image. In the background, you can see the ocean with waves crashing onto the shore. The sky appears to be clear and sunny.
<|User|>:
Describe the image in detail.
<|Assistant|>: The image shows a person standing on a beach, with their back to the camera. The person is wearing a long-sleeved shirt and pants, and has a backpack on their shoulders. They are looking out at the ocean, which is calm and blue. In the background, there are some rocks and cliffs, as well as a few seagulls flying overhead. The sky is clear and sunny, and the overall atmosphere of the image is peaceful and serene.
Obviously these responses are wrong. This model doesn't seem to work properly.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.