V100，train_xformers ,Cuda OOM

Nobody has claimed this yet.

I got an CUDA OOM in Tesla V100-PCIE 32GB * 1 with cuda 11.6 。

for some reason， i can not upload full logs 。 it Mainly is ： CUDA OOM。
my version is 0.2.32 。 and i have use train_xformers 。

my scripts is ：
torchrun --nproc_per_node=1 --master_port=20001 fastchat/train/train_xformers.py \
--model_name_or_path ~/model_weights/llama-7b \
--data_path ~/datasets/sharegpt_20230422_clean_lang_split_identity.json \
--bf16 False \
--output_dir output_vicuna_7b \
--num_train_epochs 1 \
--per_device_train_batch_size 1 \
--per_device_eval_batch_size 1 \
--gradient_accumulation_steps 2 \
--evaluation_strategy "no" \
--eval_steps 1500 \
--save_strategy "no" \
--save_steps 1500 \
--save_total_limit 8 \
--learning_rate 2e-5 \
--weight_decay 0. \
--warmup_ratio 0.04 \
--lr_scheduler_type "cosine" \
--logging_steps 1 \
--fsdp "full_shard auto_wrap" \
--fsdp_transformer_layer_cls_to_wrap 'LlamaDecoderLayer' \
--tf32 False \
--model_max_length 2048 \
--gradient_checkpointing True \
--lazy_preprocess True

how di i fix OOM ？

The issue provides no repository file or test; start by reproducing the supplied torchrun command with FastChat 0.2.32, a V100, CUDA 11.6, and the listed training options. Collect the complete logs and determine a reproducible configuration that avoids the CUDA OOM; the issue does not define a code change or test for completion.

Written by the indexing model from the issue text.