lora可以训练，但是启用qlora之后就oom

你好，我用lora正常训练的时候是可以跑的，但是当我把配置文件改成qlora的之后（也就是加了quantization_config)，就会提示显存OOM，我的默认配置是sequence_parallel_size=4， deepseed 是用的zero3+offloading，模型是qwen2 72B，配置的变动如下。

lora的配置：
llm=dict(
type=AutoModelForCausalLM.from_pretrained,
pretrained_model_name_or_path=pretrained_model_name_or_path,
trust_remote_code=True,
torch_dtype=torch.bfloat16,
attn_implementation='flash_attention_2',
# quantization_config=dict(
# type=BitsAndBytesConfig,
# load_in_4bit=True,
# load_in_8bit=False,
# llm_int8_threshold=6.0,
# llm_int8_has_fp16_weight=False,
# bnb_4bit_compute_dtype=torch.float16,
# bnb_4bit_use_double_quant=True,
# bnb_4bit_quant_type='nf4')
# ),

qlora的配置：
llm=dict(
type=AutoModelForCausalLM.from_pretrained,
pretrained_model_name_or_path=pretrained_model_name_or_path,
trust_remote_code=True,
torch_dtype=torch.float16,
# attn_implementation='flash_attention_2',
quantization_config=dict(
type=BitsAndBytesConfig,
load_in_4bit=True,
load_in_8bit=False,
llm_int8_threshold=6.0,
llm_int8_has_fp16_weight=False,
bnb_4bit_compute_dtype=torch.float16,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type='nf4')
),

qlora时报的错误如下：

Loading checkpoint shards: 100%|██████████| 37/37 [20:37<00:00, 32.48s/it]
Loading checkpoint shards: 100%|██████████| 37/37 [20:37<00:00, 33.44s/it]

Loading checkpoint shards: 100%|██████████| 37/37 [20:37<00:00, 32.49s/it]
Loading checkpoint shards: 100%|██████████| 37/37 [20:37<00:00, 33.46s/it]

Loading checkpoint shards: 100%|██████████| 37/37 [20:36<00:00, 32.49s/it]
Loading checkpoint shards: 100%|██████████| 37/37 [20:36<00:00, 33.43s/it]
Traceback (most recent call last):
File "/opt/ml/job/xtuner/tools/train.py", line 360, in
main()
File "/opt/ml/job/xtuner/tools/train.py", line 356, in main
runner.train()
File "/usr/local/lib/python3.10/dist-packages/mmengine/runner/_flexible_runner.py", line 1182, in train
self.strategy.prepare(
File "/usr/local/lib/python3.10/dist-packages/mmengine/_strategy/deepspeed.py", line 386, in prepare
self.model = self._wrap_model(model)
File "/opt/ml/job/xtuner/engine/_strategy/deepspeed.py", line 25, in _wrap_model
wrapper = super()._wrap_model(model)
File "/usr/local/lib/python3.10/dist-packages/mmengine/_strategy/deepspeed.py", line 401, in _wrap_model
engine, self.optim_wrapper.optimizer, *_ = deepspeed.initialize(
File "/usr/local/lib/python3.10/dist-packages/deepspeed/__init__.py", line 171, in initialize
engine = DeepSpeedEngine(args=args,
File "/usr/local/lib/python3.10/dist-packages/deepspeed/runtime/engine.py", line 262, in __init__
self._configure_distributed_model(model)
File "/usr/local/lib/python3.10/dist-packages/deepspeed/runtime/engine.py", line 1084, in _configure_distributed_model
self.module.to(self.device)
File "/usr/local/lib/python3.10/dist-packages/mmengine/model/base_model/base_model.py", line 208, in to
return super().to(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1145, in to
return self._apply(convert)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 797, in _apply
module._apply(fn)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 797, in _apply
module._apply(fn)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 797, in _apply
module._apply(fn)
[Previous line repeated 5 more times]
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 820, in _apply
param_applied = fn(param)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1143, in convert
return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/nn/modules.py", line 330, in to
super().to(device=device, dtype=dtype, non_blocking=non_blocking),
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 116.00 MiB (GPU 4; 39.39 GiB total capacity; 38.03 GiB already allocated; 58.38 MiB free; 38.16 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF