CHORD算法在Qwen2.5-VL-32B不work

grpo与cold start数据格式完全相同，json文件包含images、problem、answer、solution四个字段。cold start中带有think过程，grpo只有标签。但log中感觉sft过程没有学习，配置文件信息如下：

export LOG_LEVEL=DEBUG
MAX_PIXELS=200704
WANDB_API_KEY=97c188294999d80264588d15d62791cf5af3da6a
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
NPROC_PER_NODE=8
swift rlhf
--rlhf_type grpo
--model /group/40106/model_output/Qwen_96w_full/checkpoint-227652
--model_type qwen2_5_vl
--dataset /group/40106/data/antifraud_lvlm/rl_data/data/train_with_solution.json
--val_dataset /group/40106/data/antifraud_lvlm/rl_data/data/val_with_solution.json
--load_from_cache_file True
--torch_dtype bfloat16
--beta 0.0
--lora_rank 64
--max_pixels 200704
--generation_batch_size 32
--num_train_epochs 1
--max_length 4096
--per_device_train_batch_size 2
--per_device_eval_batch_size 1
--gradient_accumulation_steps 2
--chord_sft_per_device_train_batch_size 4
--chord_sft_dataset /group/40106/data/antifraud_lvlm/cold_start/data/cold_start_chord_with_solution.json
--chord_enable_phi_function True
--chord_mu_warmup_steps 0
--chord_mu_decay_steps 1000
--chord_mu_peak 0.95
--chord_mu_valley 0.05
--num_generations 8
--train_type lora
--freeze_vit true
--attn_impl flash_attention_2
--external_plugins examples/train/grpo/plugin/plugin.py
--reward_funcs antifraud_reward
--use_vllm true
--vllm_mode colocate
--vllm_gpu_memory_utilization 0.5
--vllm_max_model_len 8192
--vllm_max_lora_rank 64
--vllm_tensor_parallel_size 4
--max_completion_length 1024
--overlong_filter true
--offload_optimizer true
--offload_model true
--sleep_level 1
--save_steps 1000
--eval_steps 1000
--learning_rate 1e-6
--save_total_limit 2
--logging_steps 1
--warmup_ratio 0.05
--dataloader_num_workers 4
--deepspeed zero3
--move_model_batches 20
--use_liger_kernel false
--output_dir /group/40106/model_output_rl/qwen25_chord
--log_completions true

Start by reproducing the swift rlhf command with the provided Qwen2.5-VL configuration and inspect the training logs around the CHORD SFT stage. Compare the train_with_solution.json and cold_start_chord_with_solution.json inputs and review examples/train/grpo/plugin/plugin.py; done means identifying why the SFT component is not learning and documenting a verified fix.