how to load the ckpt of the Lora training?

Nobody has claimed this yet.

Hi, I run the following Lora training script:
```

deepspeed fastchat/train/train_lora.py \
--deepspeed configs/deepspeed_zero3.json \
--lora_r 8 \
--lora_alpha 16 \
--lora_dropout 0.05 \
--model_name_or_path huggyllama/llama-7b \
--data_path data/fastchat_data/0-sharegpt.jsonl \
--bf16 True \
--output_dir ./ckpt/llama-7b-lora-sharegpt \
--num_train_epochs 6 \
--per_device_train_batch_size 6 \
--per_device_eval_batch_size 4 \
--gradient_accumulation_steps 4 \
--evaluation_strategy "steps" \
--eval_steps 1500 \
--save_strategy "steps" \
--save_steps 500 \
--save_total_limit 100 \
--learning_rate 2e-5 \
--weight_decay 0. \
--warmup_ratio 0.03 \
--lr_scheduler_type "cosine" \
--logging_steps 1 \
--model_max_length 2048 \
--report_to "wandb" \
--gradient_checkpointing True \
--dataloader_num_workers 4
```

during training, I get some ckpts.
For example:

May I ask how can I load the full model from the ckpt?

Start by reading train_lora.py and the checkpoint layout produced under ./ckpt/llama-7b-lora-sharegpt. Trace how the script saves training checkpoints, then document the supported steps for loading the full model from one. Done means a newcomer can follow the documented procedure using the checkpoint shown in the issue.

Written by the indexing model from the issue text.