QLoRA fin-tunes a custom model with 4-bits, and inference the video, then we got :

RuntimeError: Error(s) in loading state_dict for Videollama2MistralForCausalLM:
size mismatch for model.mm_projector.readout.0.weight: copying a param with shape torch.Size([4096, 4096]) from checkpoint, the shape in current model is torch.Size([8388608, 1]).
size mismatch for model.mm_projector.readout.2.weight: copying a param with shape torch.Size([4096, 4096]) from checkpoint, the shape in current model is torch.Size([8388608, 1]).

Start by reproducing the QLoRA 4-bit custom-model load and video inference path described in the issue. Compare the checkpoint and current model shapes for model.mm_projector.readout.0.weight and readout.2.weight; done means the fine-tuned checkpoint loads without size-mismatch errors and inference completes.

Written by the indexing model from the issue text.