Using train_with_template on mistral end up in a model with a loop

I use train_with_template.py with mistralai/Mistral-7B-Instruct-v0.2

torchrun --nproc_per_node=2 --master_port=20001 fastchat/train/train_with_template.py \
    --model_name_or_path mistralai/Mistral-7B-Instruct-v0.2 \
    --data_path data/dummy_conversation.json \
    --bf16 True \
    --output_dir mistral-7b \
    --num_train_epochs 3 \
    --per_device_train_batch_size 2 \
    --per_device_eval_batch_size 2 \
    --gradient_accumulation_steps 16 \
    --evaluation_strategy "no" \
    --save_strategy "steps" \
    --save_steps 1200 \
    --save_total_limit 10 \
    --learning_rate 2e-5 \
    --weight_decay 0. \
    --warmup_ratio 0.03 \
    --lr_scheduler_type "cosine" \
    --logging_steps 1 \
    --fsdp "full_shard auto_wrap" \
    --fsdp_transformer_layer_cls_to_wrap 'MistralDecoderLayer' \
    --tf32 True \
    --model_max_length 2048 \
    --gradient_checkpointing True \
    --lazy_preprocess False

Then I run the model:

python3 -m fastchat.serve.cli --model-path mistral-7b-0103/ --num-gpus 2 --debug

I get:

Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:02<00:00,  2.29it/s]
MistralForCausalLM(
  (model): MistralModel(
    (embed_tokens): Embedding(32000, 4096)
    (layers): ModuleList(
      (0-31): 32 x MistralDecoderLayer(
        (self_attn): MistralSdpaAttention(
          (q_proj): Linear(in_features=4096, out_features=4096, bias=False)
          (k_proj): Linear(in_features=4096, out_features=1024, bias=False)
          (v_proj): Linear(in_features=4096, out_features=1024, bias=False)
          (o_proj): Linear(in_features=4096, out_features=4096, bias=False)
          (rotary_emb): MistralRotaryEmbedding()
        )
        (mlp): MistralMLP(
          (gate_proj): Linear(in_features=4096, out_features=14336, bias=False)
          (up_proj): Linear(in_features=4096, out_features=14336, bias=False)
          (down_proj): Linear(in_features=14336, out_features=4096, bias=False)
          (act_fn): SiLU()
        )
        (input_layernorm): MistralRMSNorm()
        (post_attention_layernorm): MistralRMSNorm()
      )
    )
    (norm): MistralRMSNorm()
  )
  (lm_head): Linear(in_features=4096, out_features=32000, bias=False)
)
[INST]: Hello
[/INST]: How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How

{'conv_template': 'mistral', 'prompt': '[INST] Hello [/INST]', 'outputs': 'How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How can I help you today? How', 'speed (token/s)': 58.84}

There might be a problem with the stop token or something?

Reproduce the behavior with fastchat/train/train_with_template.py and fastchat.serve.cli using the commands in the report. Inspect the Mistral conversation template and stop-token handling first, then verify that generation terminates normally instead of repeating the same response.