DeepSpeed-VisualChat Tensor shape mismatch

When I use the CLI interface I get some errors, is this a bug?

Enter image pathes, seperate by space (only support one image per time for now) (type 'na' for empty image): /root/ljh/lab/DeepSpeedExamples/applications/DeepSpeed-VisualChat/chat/bridie.jpg
Enter input (type 'quit' to exit, 'clear' to clean memory):
Traceback (most recent call last):
File "/root/ljh/lab/DeepSpeedExamples/applications/DeepSpeed-VisualChat/chat/chat.py", line 169, in
main()
File "/root/ljh/lab/DeepSpeedExamples/applications/DeepSpeed-VisualChat/chat/chat.py", line 155, in main
generate_output = model.generate(image_tensor, input_ids, generation_length=args.generation_length_per_round)
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/root/ljh/lab/DeepSpeedExamples/applications/DeepSpeed-VisualChat/utils/model/modeling_dsvl.py", line 366, in generate
output = self.lang_decoder.generate(input_ids=None,
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py", line 1648, in generate
return self.sample(
File "/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py", line 2730, in sample
outputs = self(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1505, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1514, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 164, in new_forward
output = module._old_forward(*args, **kwargs)
File "/root/ljh/lab/DeepSpeedExamples/applications/DeepSpeed-VisualChat/utils/model/third_party_model/hf_model/modeling_llama.py", line 907, in forward
logits = self.lm_head(hidden_states)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1505, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1514, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 159, in new_forward
args, kwargs = module._hf_hook.pre_forward(module, *args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 286, in pre_forward
set_module_tensor_to_device(
File "/usr/local/lib/python3.10/dist-packages/accelerate/utils/modeling.py", line 285, in set_module_tensor_to_device
raise ValueError(
ValueError: Trying to set a tensor of shape torch.Size([32000, 8192]) in "weight" (which has shape torch.Size([32003, 8192])), this look incorrect.

Reproduce the CLI failure using the command and image path from the report, then inspect chat/chat.py around lines 155 and 169 and utils/model/modeling_dsvl.py around generate at line 366. Follow the call into utils/model/third_party_model/hf_model/modeling_llama.py around line 907 and compare the reported tensor shapes; done means the CLI no longer raises this mismatch or the expected configuration issue is documented.