[Feature] 请问是否可以用qwen2.5-vl的video frame的相邻帧优化
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 748
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 54
Description
*### Motivation
如题,参考https://lmdeploy.readthedocs.io/zh-cn/latest/multi_modal/qwen2_5_vl.html
进行视频帧推理时,是否能正常使用qwen2.5-vl的video frame相邻帧优化?还是把所有帧都作为单图token进行使用。
我的代码如下:
`
question = ''
*# 视频封面图
cover_img = None
question = question + f'cover image:\n{IMAGE_TOKEN}\n\n'
*# 视频帧
imgs = []
question = question + 'visual frames:\n'
for i in range(len(imgs)):
question = question + f'Frame{i+1}: {IMAGE_TOKEN}\n'
*# ques
question += 'What are the cover and visuals of this video talking about?'
*# content
content = [{'type': 'text', 'text': question}]
content.append({'type': 'image_url', 'image_url': {"url": f"{cover_img}"}})
for img in imgs:
content.append({'type': 'image_url', 'image_url': {'url': f"{img}"}})
messages = [dict(role='user', content=content)]
`
如果上述无法实现我的目标作用的话,是否有其它wiki参考进行实现?
期待您的回复,感谢!
### Related resources
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.