huggingface / huggingface/chat-ui
IDEFICS works with image URLs but not "upload image"
- Dominant language
- TypeScript
- Stars
- 11k
- Forks
- 1.7k
- Avg merge
- 21h 24m
- Merged PRs (30d)
- 56
Description
Hi there,
I am hosting IDEFICS on TGI locally and hitting it via Chat-UI. When I go to a URL, it is able to correctly work with images, as seen here:
However, when I use the "upload image" feature, or the drag and drop, it looks like it is ignoring the image:
Here is my model config (using the latest main branch of chat-ui 0.7.0):
` {
"name": "HuggingFaceM4/idefics-9b-instruct",
"endpoints":[{"type":"tgi","url":"http://INTERNAL URL:/generate_stream"}],
"multimodal" : true,
"description": "IDEFICS is the new multimodal model by Hugging Face.",
"preprompt": "",
"chatPromptTemplate" : "{{#each messages}}{{#ifUser}}User: {{content}}{{/ifUser}}\nAssistant: {{#ifAssistant}}{{content}}\n{{/ifAssistant}}{{/each}}",
"parameters": {
"temperature": 0.1,
"top_p": 0.95,
"repetition_penalty": 1.2,
"top_k": 12,
"truncate": 1000,
"max_new_tokens": 1024,
"stop": ["", "User:", "\nUser:"]
}
}`
And here is my TGI command args from docker compose (using 1.3.4 container for TGI):
` --sharded true --num-shard 2 --dtype float16 --model-id HuggingFaceM4/idefics-9b-instruct --max-total-tokens 2048 --max-input-length 1000 --max-batch-prefill-tokens 2048`
Here is the [image URL](https://commons.wikimedia.org/wiki/Dog#/media/File:Flat_Coated_Retriever_-_black.jpg) for reference.
A gigantic thank you for putting together such wonderful tools!! And thank you in advance for your help.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.