[Feature] Support for CogVLM2-Video-LLama3-Chat in TorchEngine
Open
backlog
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 748
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 54
Description
### Motivation
Hi team,
ZhipuAI just released their multi-modal model `CogVLM2-Video-LLama3-Chat`. Can we support its serving with TorchEngine? It seems that they use a new causal model architecture called `CogVLMVideoForCausalLM`.
I’m really excited to try out this model with lmdeploy.
Here's the link for reference: [CogVLM2-Video-LLama3-Chat Model](https://modelscope.cn/models/ZhipuAI/cogvlm2-video-llama3-chat/file/view/master?fileName=modeling_cogvlm.py&status=1).
Thanks!
### Related resources
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.