[Feature] 可以考虑支持THUDM/cogvlm2-llama3-chinese-chat-19B-int4模型吗

### Motivation

使用python脚本
```python
from lmdeploy import pipeline
from lmdeploy.vl import load_image

if __name__ == "__main__":
pipe = pipeline('THUDM/cogvlm2-llama3-chinese-chat-19B-int4')

image = load_image('/home/ky/project/imdeploy/demo.jpg')
response = pipe(('describe this image', image))
print(response)
```

启动推理的时候,报错:Fetching 15 files: 100%|█████████████████████████████████████████████| 15/15 [00:00<00:00, 302473.85it/s]
2024-11-24 18:13:20,783 - lmdeploy - WARNING - archs.py:53 - Fallback to pytorch engine because `/home/ky/.cache/huggingface/hub/models--THUDM--cogvlm2-llama3-chinese-chat-19B-int4/snapshots/7863e362174f4718c2fe9cba4befd0b580a3194f` not supported by turbomind engine.
/home/ky/miniconda3/envs/imdeploy/lib/python3.12/site-packages/accelerate/utils/modeling.py:1513: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
return torch.load(checkpoint_file, map_location=torch.device("cpu"))
Traceback (most recent call last):
File "/home/ky/project/imdeploy/cogvlm2.py", line 6, in
pipe = pipeline('THUDM/cogvlm2-llama3-chinese-chat-19B-int4')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ky/miniconda3/envs/imdeploy/lib/python3.12/site-packages/lmdeploy/api.py", line 81, in pipeline
return pipeline_class(model_path,
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ky/miniconda3/envs/imdeploy/lib/python3.12/site-packages/lmdeploy/serve/vl_async_engine.py", line 24, in __init__
self.vl_encoder = ImageEncoder(model_path,
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ky/miniconda3/envs/imdeploy/lib/python3.12/site-packages/lmdeploy/vl/engine.py", line 90, in __init__
self.model = load_vl_model(model_path, backend_config=backend_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ky/miniconda3/envs/imdeploy/lib/python3.12/site-packages/lmdeploy/vl/model/builder.py", line 65, in load_vl_model
return module(**kwargs)
^^^^^^^^^^^^^^^^
File "/home/ky/miniconda3/envs/imdeploy/lib/python3.12/site-packages/lmdeploy/vl/model/base.py", line 31, in __init__
self.build_model()
File "/home/ky/miniconda3/envs/imdeploy/lib/python3.12/site-packages/lmdeploy/vl/model/cogvlm.py", line 63, in build_model
load_checkpoint_and_dispatch(
File "/home/ky/miniconda3/envs/imdeploy/lib/python3.12/site-packages/accelerate/big_modeling.py", line 613, in load_checkpoint_and_dispatch
load_checkpoint_in_model(
File "/home/ky/miniconda3/envs/imdeploy/lib/python3.12/site-packages/accelerate/utils/modeling.py", line 1780, in load_checkpoint_in_model
set_module_tensor_to_device(
File "/home/ky/miniconda3/envs/imdeploy/lib/python3.12/site-packages/accelerate/utils/modeling.py", line 286, in set_module_tensor_to_device
raise ValueError(
ValueError: Trying to set a tensor of shape torch.Size([4816896, 1]) in "weight" (which has shape torch.Size([5376, 1792])), this looks incorrect

### Related resources

_No response_

### Additional context

_No response_