Finetuning with LLaMA-Efficient-Tuning and deploying with fastchat， but get poor result

Nobody has claimed this yet.

I have fine-tuned the Baichuan2 model using another framework and deployed it using fastchat. However, I encountered the following issue: the accuracy on fastchat decreased. Here is the deployment code I used:

python3 -m fastchat.serve.cli --model-path /backup/baichuan/912/baichuan2-fintuned-50b
I would like to know which parameters I should modify to solve this problem and I would appreciate guidance on parameter selection if possible. If it is not possible to solve this issue, does it mean that I can only achieve the same results by fine-tuning and deploying within the fastchat framework?

The fine-tuning project I used is available at:
https://github.com/hiyouga/LLaMA-Efficient-Tuning/blob/main/README_zh.md
By using the local deployment demo provided by this project, I obtained consistent results (overfitting with 99%+ accuracy on the training set, and good generated outputs when deploying the model).

Start with the reported fastchat.serve.cli command and compare it with the local deployment demo from the linked LLaMA-Efficient-Tuning README. Check which model-loading and generation parameters differ, then reproduce the accuracy drop with the Baichuan2 fine-tuned model. Done means identifying a supported parameter or a reproducible compatibility limitation.

Written by the indexing model from the issue text.