NotImplementedError: Cannot copy out of meta tensor; no data!

Nobody has claimed this yet.

Hello,I used Qlora to train,but I get an error:
NotImplementedError: Cannot copy out of meta tensor; no data!

requriements.txt:
```
peft @ file:///root/peft
torch==1.13.1+cu116
torchaudio==0.13.1+cu116
torchvision==0.14.1+cu116
transformers==4.28.1
deepspeed==0.9.4
flash-attn==0.2.0
```

This my train code:

```
CUDA_VISIBLE_DEVICES=0 deepspeed fastchat/train/train_lora.py \
--model_name_or_path ../vicuna-7b \
--lora_r 8 \
--lora_alpha 16 \
--lora_dropout 0.05 \
--data_path ./data/dummy_conversation.json \
--bf16 True \
--output_dir ./checkpoints \
--num_train_epochs 3 \
--per_device_train_batch_size 4 \
--per_device_eval_batch_size 4 \
--gradient_accumulation_steps 1 \
--evaluation_strategy "no" \
--save_strategy "steps" \
--save_steps 1200 \
--save_total_limit 100 \
--learning_rate 2e-5 \
--weight_decay 0. \
--warmup_ratio 0.03 \
--lr_scheduler_type "cosine" \
--logging_steps 1 \
--tf32 True \
--model_max_length 2048 \
--q_lora True \
--deepspeed playground/deepspeed_config_s2.json
```

Can you help me,please?
@merrymercy

Start by reproducing the command for fastchat/train/train_lora.py with the listed requirements and playground/deepspeed_config_s2.json. Trace where the “Cannot copy out of meta tensor” error occurs during QLoRA training. Done means the provided training command starts without this error.

Written by the indexing model from the issue text.