Fine tune llama 2

Two questions regarding llama 2 fine tuning:

1. it seems the prompt templates defaults to vincuna and cannot overwritten according to the following code:
   https://github.com/lm-sys/FastChat/blob/cfc73bf3e13c22ded81e89675e0d7b228cf4b342/fastchat/train/train.py#L85

2. when I hard code to llama-2, I got training loss being 0 forever, while the loss is normal when switching back to vicuna. Could be related to 'llama-2' prompt template?

The code to reproduce the result:
python3 train_lora.py \ --model_name_or_path meta-llama/Llama-2-7b \ --lora_r 16 \ --lora_alpha 32 \ --lora_dropout 0.05 \ --data_path data/dummy_conversation.json \ --output_dir /llama-2-output \ --num_train_epochs 4 \ --fp16 True \ --per_device_train_batch_size 4 \ --per_device_eval_batch_size 8 \ --gradient_accumulation_steps 2 \ --evaluation_strategy "no" \ --eval_steps 100\ --save_strategy "steps" \ --save_steps 100\ --save_total_limit 2 \ --learning_rate 2e-5 \ --weight_decay 0. \ --warmup_ratio 0.03 \ --lr_scheduler_type "cosine" \ --logging_strategy "steps" \ --logging_steps 1 \ --tf32 True \ --model_max_length 2048 \ --q_lora False \ --gradient_checkpointing True \ --flash_attn False \ --lazy_preprocess True

Start with fastchat/train/train.py around the prompt-template selection referenced at line 85, then inspect train_lora.py and data/dummy_conversation.json. Run the supplied command with the Vicuna and llama-2 templates and compare how prompts and labels are produced. Done means the intended template can be selected and the llama-2 run no longer reports a permanently zero training loss.