如何将xtuner微调得到的llava模型转换成lmdeploy可部署的格式
- Dominant language
- Python
- Stars
- 5.2k
- Forks
- 448
- Avg merge
- 3d 15h
- Merged PRs (30d)
- 26
Description
目前,我根据教程可用下面的命令与微调后的模型进行对话
```
xtuner chat /root/internlm2-chat-1_8b \
--visual-encoder /root/share/new_models/openai/clip-vit-large-patch14-336 \
--llava /root/Tutorial/xtuner/llava/llava_data/iter_1200_hf \
--prompt-template internlm2_chat \
--image /root/Tutorial/xtuner/llava/llava_data/test_img/oph.jpg
```
现在我想用lmdeploy对微调后模型进行部署,类似下面这样
```
import gradio as gr
from lmdeploy import pipeline, TurbomindEngineConfig
backend_config = TurbomindEngineConfig(session_len=8192) # 图片分辨率较高时请调高session_len
pipe = pipeline('/share/new_models/liuhaotian/llava-v1.6-vicuna-7b', backend_config=backend_config)
def model(image, text):
if image is None:
return [(text, "请上传一张图片。")]
else:
response = pipe((text, image)).text
return [(text, response)]
demo = gr.Interface(fn=model, inputs=[gr.Image(type="pil"), gr.Textbox()], outputs=gr.Chatbot())
demo.launch()
```
请问我该如何对微调后的模型进行转换
Contributor guide
Research direction
Start with the provided xtuner chat command and the lmdeploy Python pipeline example, checking how the visual encoder and iter_1200_hf model are represented. Determine whether the repository documents a conversion or deployment entry point for this fine-tuned LLaVA model. Done means a documented, reproducible conversion path or a clear statement of the required supported workflow.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devops, machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100