failed to convert convert llava-llama3 model to hf format

In the project: https://github.com/InternLM/xtuner/tree/main/xtuner/configs/llava/llama3_8b_instruct_clip_vit_large_p14_336, it gives an examples how to convert llava-llama3 model to hf format:

`
python ./convert_xtuner_weights_to_hf.py --text_model_id ./iter_39620_xtuner --vision_model_id ./iter_39620_visual_encoder --projector_weight ./iter_39620_xtuner/projector/model.safetensors --save_path ./iter_39620_llava`

I follow it in this way:
1. I train llava-llama3 model with single gpu with llava_llama3_8b_instruct_qlora_clip_vit_large_p14_336_e1_gpu1_finetune.py, and get the pth named lama3_llava_pth;
2. convert llama3_llava_pth to hugggingface format named llama3_llava_pth/hf;
3. I merge the llm_adapter and original llama3 to new llama3 with xtuner merge tool, and create ./llama3_llava_pth/merge
4. I try to convert format
`CUDA_VISIBLE_DEVICES=4 python ./xtuner/configs/llava/llama3_8b_instruct_clip_vit_large_p14_336/convert_xtuner_weights_to_hf.py --text_model_id ./llama3_llava_pth/merge --vision_model_id ${vit} --projector_weight llama3_llava_pth/hf/projector --save_path ./llama3_llava_pth/LLava_format`

however I get the error
```
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization.
The tokenizer class you load from this checkpoint is 'PreTrainedTokenizerFast'.
The class this function is called from is 'LlamaTokenizerFast'.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Traceback (most recent call last):
File "/Data_PHD/****/projects/xtuner/./xtuner/configs/llava/llama3_8b_instruct_clip_vit_large_p14_336/convert_xtuner_weights_to_hf.py", line 140, in
main()
File "/Data_PHD/****/projects/xtuner/./xtuner/configs/llava/llama3_8b_instruct_clip_vit_large_p14_336/convert_xtuner_weights_to_hf.py", line 135, in main
convert_to_hf(args.text_model_id, args.vision_model_id,
File "/Data_PHD/****/projects/xtuner/./xtuner/configs/llava/llama3_8b_instruct_clip_vit_large_p14_336/convert_xtuner_weights_to_hf.py", line 61, in convert_to_hf
model = LlavaForConditionalGeneration(config)
File "/Data_PHD/****/anaconda3_35/envs/XTUNER/lib/python3.10/site-packages/transformers/models/llava/modeling_llava.py", line 244, in __init__
self.multi_modal_projector = LlavaMultiModalProjector(config)
File "/Data_PHD/****/anaconda3_35/envs/XTUNER/lib/python3.10/site-packages/transformers/models/llava/modeling_llava.py", line 93, in __init__
self.linear_1 = nn.Linear(config.vision_config.hidden_size, config.text_config.hidden_size, bias=True)
File "/Data_PHD/****/anaconda3_35/envs/XTUNER/lib/python3.10/site-packages/transformers/configuration_utils.py", line 263, in __getattribute__
return super().__getattribute__(key)
AttributeError: 'CLIPConfig' object has no attribute 'hidden_size'`
```

is there any error in my steps, thank you for you answer!