InternLM / InternLM/xtuner

multi image inputs supports for xtuner chat in llava-llama3?

Open
#655 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
5.2k
Forks
448
Avg merge
3d 15h
Merged PRs (30d)
26

Description

- cmd:
`xtuner chat LLM-Research/Meta-Llama-3-8B-Instruct \
--visual-encoder ./clip-vit-large-patch14-336 \
--llava ./LLM-Research/llava-llama-3-8b \
--prompt-template llama3_chat \
--image ./test001.png`

- question:
trained multimodal model can only input **one image** at one time , is there any method to support multi image & queries at one time?
such as following:

```
double enter to end input (EXIT: exit chat, RESET: reset history) >>> **image input**: xxx/test.jpg or None

double enter to end input (EXIT: exit chat, RESET: reset history) >>> **query:** describe this images.

xxxxxxxxxxxxxx

double enter to end input (EXIT: exit chat, RESET: reset history) >>>
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.