InternLM / InternLM/xtuner

如何将xtuner微调得到的llava模型转换成lmdeploy可部署的格式

Open
#742 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
5.2k
Forks
448
Avg merge
3d 15h
Merged PRs (30d)
26

Description

目前,我根据教程可用下面的命令与微调后的模型进行对话
```
xtuner chat /root/internlm2-chat-1_8b \
--visual-encoder /root/share/new_models/openai/clip-vit-large-patch14-336 \
--llava /root/Tutorial/xtuner/llava/llava_data/iter_1200_hf \
--prompt-template internlm2_chat \
--image /root/Tutorial/xtuner/llava/llava_data/test_img/oph.jpg
```

现在我想用lmdeploy对微调后模型进行部署,类似下面这样
```
import gradio as gr
from lmdeploy import pipeline, TurbomindEngineConfig

backend_config = TurbomindEngineConfig(session_len=8192) # 图片分辨率较高时请调高session_len
pipe = pipeline('/share/new_models/liuhaotian/llava-v1.6-vicuna-7b', backend_config=backend_config)

def model(image, text):
if image is None:
return [(text, "请上传一张图片。")]
else:
response = pipe((text, image)).text
return [(text, response)]

demo = gr.Interface(fn=model, inputs=[gr.Image(type="pil"), gr.Textbox()], outputs=gr.Chatbot())
demo.launch()
```

请问我该如何对微调后的模型进行转换

Contributor guide

Open the contributing guide

Research direction

Start with the provided xtuner chat command and the lmdeploy Python pipeline example, checking how the visual encoder and iter_1200_hf model are represented. Determine whether the repository documents a conversion or deployment entry point for this fine-tuned LLaVA model. Done means a documented, reproducible conversion path or a clear statement of the required supported workflow.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devops, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.