Error while loading Mixtral based SFT MoE model VideoLLaMA2-8x7B: SafetensorError: Error while deserializing header: InvalidHeaderDeserialization

Hi Team,

When I am loading the Mixtral-based SFT MoE model 'DAMO-NLP-SG/VideoLLaMA2-8x7B' using the same inference code provided in the README.md, the following error is raised:

```
Traceback (most recent call last):
File "/home/admin/apoorv/development/VideoLLaMA2/playground.py", line 33, in
inference()
File "/home/admin/apoorv/development/VideoLLaMA2/playground.py", line 27, in inference
model, processor, tokenizer = model_init(model_path)
File "/home/admin/apoorv/development/VideoLLaMA2/videollama2/__init__.py", line 17, in model_init
tokenizer, model, processor, context_len = load_pretrained_model(model_path, None, model_name, **kwargs)
File "/home/admin/apoorv/development/VideoLLaMA2/videollama2/model/__init__.py", line 180, in load_pretrained_model
model = Videollama2MixtralForCausalLM.from_pretrained(model_path, low_cpu_mem_usage=True, config=config, **kwargs)
File "/home/admin/.conda/envs/vl2/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3838, in from_pretrained
) = cls._load_pretrained_model(
File "/home/admin/.conda/envs/vl2/lib/python3.10/site-packages/transformers/modeling_utils.py", line 4278, in _load_pretrained_model
state_dict = load_state_dict(shard_file, is_quantized=is_quantized)
File "/home/admin/.conda/envs/vl2/lib/python3.10/site-packages/transformers/modeling_utils.py", line 516, in load_state_dict
with safe_open(checkpoint_file, framework="pt") as f:
safetensors_rust.SafetensorError: Error while deserializing header: InvalidHeaderDeserialization
```

I tried to find the reason for this and came across the following issues where the main reason was that the file/weights saved were not done correctly and had empty dictionaries which the safetensors can't handle.
https://github.com/huggingface/transformers/issues/27397#issuecomment-1806063673

To solve this, there are some changes to be implemented before saving the model checkpoint by you guys:
1. https://github.com/huggingface/transformers/issues/27397
3. https://discuss.huggingface.co/t/safetensors-rust-safetenserror-while-deserializing-header-invalidheaderdeserialization/68831
4. https://github.com/slai-labs/get-beam/issues/94
2. https://huggingface.co/TheBloke/deepseek-coder-33B-instruct-AWQ/discussions/1