InternLM / InternLM/lmdeploy

transform部署和lmdeploy部署结果不一致

Open
#3,565 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
8.1k
Forks
748
Avg merge
6d 2h
Merged PRs (30d)
54

Description

### **transformer部署**
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info
import time

start_time = time.perf_counter()
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
"/data03/public/mllm/Qwen/Qwen2.5-VL-32B-Instruct/", torch_dtype="auto", device_map="auto"
)

processor = AutoProcessor.from_pretrained("/data03/public/mllm/Qwen/Qwen2.5-VL-32B-Instruct/")

min_pixels = 256*28*28
max_pixels = 12845056
processor = AutoProcessor.from_pretrained("/data03/public/mllm/Qwen/Qwen2.5-VL-32B-Instruct/", min_pixels=min_pixels, max_pixels=max_pixels)

messages = [
{"role": "system",
"content":"你是一个专业的安全检查员,专注于分析施工场景中的安全规范,尤其是安全绳的系挂是否符合标准。"},
{
"role": "user",
"content": [
{
"type": "image",
"image": "/data02/mahg/project/qwen2.5_vl_test/001.png",
},
{"type": "text", "text": "##任务描述:\n请分析图像中的施工人员是否正确系挂安全绳,并给出推理步骤。\n##背景知识:\n安全绳有5种状态:安全绳打结、安全绳缠绕、安全绳垂悬、安全绳未拉伸、安全绳拉伸\n安全>绳挂钩有2种状态:挂钩系挂、挂钩未系挂\n##安全绳正确系挂推理步骤:\n1、首先判断安全绳钩子是否存在。如果存在,转到步骤2;否则转到步骤3\n2、判断挂钩是否系挂在牢固的外部锚点,如果是则无隐患。若悬空或系挂临时结构(
如手持),视为有隐患。结束推理步骤\n3、判断安全绳是否存在。如果存在,转到步骤4;否则视为有隐患并结束推理步骤\n4、判断安全绳状态。若安全绳打结、缠绕、垂悬、未拉伸,视为有隐患;若拉伸,视为无隐患\n##重点排除>背景干扰:\n注意区分安全绳与其他类似线条(如脚手架钢管、防坠网绳索、钢筋),避免误判。\n##输出格式:\n以JSON格式输出,包含\"conclusion\"、\"reason\"两个字段,如果有隐患则\"conclusion\"字段值为\"True\",否则值>为\"False\"。\"reason\"字段>值为具体的判断依据(例如:\"安全绳挂钩未挂接在脚手架横杆上等固定物体上\"或\"安全绳未识别到\")"}
],
}
]

text = processor.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True
)
image_inputs, video_inputs = process_vision_info(messages)
inputs = processor(
text=[text],
images=image_inputs,
videos=video_inputs,
padding=True,
return_tensors="pt",
)
inputs = inputs.to(model.device)

generated_ids = model.generate(**inputs, max_new_tokens=12800)
generated_ids_trimmed = [
out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
end_time = time.perf_counter()

print(f"耗时: {end_time - start_time:.2f} 秒")
print(output_text)

![Image](https://github.com/user-attachments/assets/f69572f9-128d-42f2-9fb5-fdb5b543dd89)

**### lmdeploy部署**
`import time
from lmdeploy import pipeline, GenerationConfig
from lmdeploy.vl import load_image
start_time = time.perf_counter()
pipe = pipeline('/data03/public/mllm/Qwen/Qwen2.5-VL-32B-Instruct/', log_level='INFO')

min_pixels = 256*28*28
max_pixels = 12845056
messages = [
{"role": "system",
"content":"你是一个专业的安全检查员,专注于分析施工场景中的安全规范,尤其是安全绳的系挂是否符合标准。"},
{
"role": "user",
"content": [
{
"type": "image",
"image_url": {"url":"/data02/mahg/project/qwen2.5_vl_test/001.png","mix_pixels":min_pixels,"max_pixels":max_pixels},
},
{"type": "text", "text": "##任务描述:\n请分析图像中的施工人员是否正确系挂安全绳,并给出推理步骤。\n安全绳有5种状态:安全绳打结、安全绳缠绕、安全绳垂悬、安全绳未拉伸、安全绳拉伸\n安全绳挂钩有2种状>态:挂钩系挂、挂钩未系挂\n##安全绳正确系挂推理步骤:\n1、首先判断安全绳钩子是否存在。如果存在,转到步骤2;否则转到步骤3\n2、判断挂钩是否系挂在牢固的外部锚点,如果是则无隐患。若悬空或系挂临时结构(如手持),视>为有隐患。结束推理步骤\n3、判断安全绳是否存在。如果存在,转到步骤4;否则视为有隐患并结束推理步骤\n4、判断安全绳状态。若安全绳打结、缠绕、垂悬、未拉伸,视为有隐患;若拉伸,视为无隐患\n##重点排除背景干扰:\n注
意区分安全绳与其他类似线条(如脚手架钢管、防坠网绳索、钢筋),避免误判。\n##输出格式:\n以JSON格式输出,包含\"conclusion\"、\"reason\"两个字段,如果有隐患则\"conclusion\"字段值为\"True\",否则值为\"False\"。\"reason\"字段>值为具体的判断依据(例如:\"安全绳挂钩未挂接在脚手架横杆上等固定物体上\"或\"安全绳未识别到\")"}
],
}
]
out = pipe(messages,GenerationConfig(max_new_tokens=12800))
pipe.close()
end_time = time.perf_counter()
print(f"耗时: {end_time- start_time:.2f} 秒")
print(out.text)`

![Image](https://github.com/user-attachments/assets/873d6f46-d3c9-4cee-931b-d28f73d41a4c)

可以看到两个结果不一样,lmdeploy少了推理过程,请我是我哪块配置有问题吗

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.