Training Loss Stuck at Zero in VideoLLaMA2.1-7B-AV

Hi,

I've been running `va_joint.sh` with the following command:

```bash
/videollama2/train.py --deepspeed scripts/zero2.json --model_type videollama2_qwen2 --model_path DAMO-NLP-SG/VideoLLaMA2.1-7B-AV --data_folder /data/valor32k/videos --data_path datasets/valor32kgpt4o/combined_dataset_train_mcq.json --vision_tower google/siglip-so400m-patch14-384 --audio_tower videollama2/av-weight/VideoLLaMA2.1-7B-AV/audio_tower.bin --pretrain_mm_mlp_adapter_a videollama2/av-weight/VideoLLaMA2.1-7B-AV/mm_projector_a.bin --mm_projector_type stc_connector_v35 --mm_projector_a_type mlp2x_gelu --va True --tune_audio_tower True --tune_adapter_llm True --tune_mm_mlp_adapter_a True --mm_vision_select_layer -2 --image_aspect_ratio pad --num_frames 16 --bf16 True --tf32 False --fp16 False --output_dir work_dirs/audio_visual_stage3_qwen2/VideoLLaMA2.1-7B-AV --num_train_epochs 2 --per_device_train_batch_size 2 --per_device_eval_batch_size 4 --gradient_accumulation_steps 8 --evaluation_strategy no --save_strategy steps --save_steps 2000 --save_total_limit 2 --learning_rate 2e-5 --weight_decay 0. --warmup_ratio 0.03 --lr_scheduler_type cosine --logging_steps 1 --model_max_length 2048 --gradient_checkpointing True --dataloader_num_workers 4 --lazy_preprocess True --report_to wandb --run_name audio_visual_stage3_qwen2
```

The training starts, but the loss remains at 0. Additionally, `grad_norm` is `nan`, which suggests something might be wrong with gradient updates.

Here's the log output:

```
(lm_head): Linear(in_features=3584, out_features=152064, bias=False)
)
Formatting inputs... Skip in lazy mode
{'loss': 42.9958, 'grad_norm': nan, 'learning_rate': 2.38e-07, 'epoch': 0.0}
{'loss': 0.0, 'grad_norm': nan, 'learning_rate': 4.76e-07, 'epoch': 0.0}
{'loss': 0.0, 'grad_norm': nan, 'learning_rate': 7.14e-07, 'epoch': 0.0}
...
{'loss': 0.0, 'grad_norm': nan, 'learning_rate': 3.33e-06, 'epoch': 0.01}
```

It seems like the model is not updating properly.

Any insights on what could be causing this issue or suggestions for debugging would be greatly appreciated. Thanks!

Start with va_joint.sh, train.py, scripts/zero2.json, and the combined_dataset_train_mcq.json data path used in the command. Reproduce the run and trace how loss and grad_norm are computed after the first step; done means training produces finite, nonzero loss and gradient values instead of zero loss with NaN grad_norm.

Written by the indexing model from the issue text.