[Docs] llava-llama3的图片预处理和前向推理过程
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 748
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 54
Description
### 📚 The doc issue
感谢你们的工作,我参考官方文档:https://github.com/InternLM/lmdeploy/blob/main/docs/zh_cn/multi_modal/vl_pipeline.md,能够跑通,代码如下
```
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "3"
from lmdeploy import pipeline, ChatTemplateConfig
from lmdeploy.vl import load_image
pipe = pipeline('**, chat_template_config=ChatTemplateConfig(model_name='llama3'))
image = load_image('**')
response = pipe(('describe this image', image))
print(response)
```
代码能照常跑通,通过debug了解些细节, 比如lmdeploy/vl/model/llava.py:build_model函数会加载vlm函数.
原本以为preprocess是图像预处理, forward是llm的前向推理过程,但是在这两处打断点, 运行的时候并没有经过, 所以想问下图像预处理模块和llm的forward是在哪加载,因为我想修改图像预处理模块, 感谢您的答复.
### Suggest a potential alternative/fix
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.