deepseek-ai / deepseek-ai/DeepSeek-VL2

VL2 Small produces wrong response for an input image

Open
#83 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
5.4k
Forks
1.8k
PR merge metrics
No merged PRs in 30d

Description

model_path = "deepseek-ai/deepseek-vl2-small"
vl_chat_processor: DeepseekVLV2Processor = DeepseekVLV2Processor.from_pretrained(model_path)
tokenizer = vl_chat_processor.tokenizer
#model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map="auto")
quantization_config = QuantoConfig(weights="int8")
vl_gpt: DeepseekVLV2ForCausalLM = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype="auto", device_map="cuda:0", trust_remote_code=True, quantization_config=quantization_config)

I update the image and question in the Conversation:
(image below)
![Image](https://github.com/user-attachments/assets/f9249d97-fce2-4b05-84a0-4bd64ba8859c)

I then ask the question: Describe the image in detail.
The response I get back is as follows after 2 successive tries:

<|Assistant|>: The image shows a person standing on a beach, with their back to the camera. The person is wearing a white shirt and blue shorts, and they have a backpack on their shoulders. The beach is sandy and there are no other people visible in the image. In the background, you can see the ocean with waves crashing onto the shore. The sky appears to be clear and sunny.
<|User|>:
Describe the image in detail.

<|Assistant|>: The image shows a person standing on a beach, with their back to the camera. The person is wearing a long-sleeved shirt and pants, and has a backpack on their shoulders. They are looking out at the ocean, which is calm and blue. In the background, there are some rocks and cliffs, as well as a few seagulls flying overhead. The sky is clear and sunny, and the overall atmosphere of the image is peaceful and serene.

Obviously these responses are wrong. This model doesn't seem to work properly.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.