deepspeed memory

Describe the bug
when i offload the memory to cpu with deepspeed offloadd strategy, the 'torch.cuda.memory_reserved()' is still very high, i don't why?

[2023-11-17 11:03:46,600] [INFO] [utils.py:802:see_memory_usage] DeepSpeedZeRoOffload initialize [begin]
[2023-11-17 11:03:46,601] [INFO] [utils.py:803:see_memory_usage] MA 26.38 GB         Max_MA 26.38 GB         CA 26.71 GB         Max_CA 27 GB 
[2023-11-17 11:03:46,602] [INFO] [utils.py:810:see_memory_usage] CPU Virtual Memory:  used = 322.57 GB, percent = 17.1%
Parameter Offload: Total persistent parameters: 483840 in 105 params
[2023-11-17 11:04:06,576] [INFO] [utils.py:802:see_memory_usage] DeepSpeedZeRoOffload initialize [end]
[2023-11-17 11:04:06,577] [INFO] [utils.py:803:see_memory_usage] MA 0.58 GB         Max_MA 26.38 GB         CA 26.71 GB         Max_CA 27 GB 
[2023-11-17 11:04:06,578] [INFO] [utils.py:810:see_memory_usage] CPU Virtual Memory:  used = 322.58 GB, percent = 17.1%
[2023-11-17 11:04:07,121] [INFO] [utils.py:802:see_memory_usage] Before creating fp16 partitions
[2023-11-17 11:04:07,122] [INFO] [utils.py:803:see_memory_usage] MA 0.58 GB         Max_MA 0.58 GB         CA 26.71 GB         Max_CA 27 GB 
[2023-11-17 11:04:07,123] [INFO] [utils.py:810:see_memory_usage] CPU Virtual Memory:  used = 322.58 GB, percent = 17.1%
[2023-11-17 11:04:30,143] [INFO] [utils.py:802:see_memory_usage] After creating fp16 partitions: 2
[2023-11-17 11:04:30,145] [INFO] [utils.py:803:see_memory_usage] MA 0.58 GB         Max_MA 0.58 GB         CA 26.71 GB         Max_CA 27 GB 
[2023-11-17 11:04:30,146] [INFO] [utils.py:810:see_memory_usage] CPU Virtual Memory:  used = 367.96 GB, percent = 19.6%
[2023-11-17 11:04:30,694] [INFO] [utils.py:802:see_memory_usage] Before creating fp32 partitions
[2023-11-17 11:04:30,695] [INFO] [utils.py:803:see_memory_usage] MA 0.58 GB         Max_MA 0.58 GB         CA 26.71 GB         Max_CA 27 GB 
[2023-11-17 11:04:30,695] [INFO] [utils.py:810:see_memory_usage] CPU Virtual Memory:  used = 367.96 GB, percent = 19.6%
[2023-11-17 11:04:31,830] [INFO] [utils.py:802:see_memory_usage] After creating fp32 partitions
[2023-11-17 11:04:31,831] [INFO] [utils.py:803:see_memory_usage] MA 0.58 GB         Max_MA 0.58 GB         CA 26.71 GB         Max_CA 27 GB 
[2023-11-17 11:04:31,832] [INFO] [utils.py:810:see_memory_usage] CPU Virtual Memory:  used = 374.43 GB, percent = 19.9%
[2023-11-17 11:04:32,929] [INFO] [utils.py:802:see_memory_usage] Before initializing optimizer states
[2023-11-17 11:04:32,930] [INFO] [utils.py:803:see_memory_usage] MA 0.58 GB         Max_MA 0.58 GB         CA 26.71 GB         Max_CA 27 GB 
[2023-11-17 11:04:32,931] [INFO] [utils.py:810:see_memory_usage] CPU Virtual Memory:  used = 426.76 GB, percent = 22.7%
[2023-11-17 11:04:42,837] [INFO] [utils.py:802:see_memory_usage] After initializing optimizer states
[2023-11-17 11:04:42,838] [INFO] [utils.py:803:see_memory_usage] MA 0.58 GB         Max_MA 0.58 GB         CA 26.71 GB         Max_CA 27 GB 
[2023-11-17 11:04:42,839] [INFO] [utils.py:810:see_memory_usage] CPU Virtual Memory:  used = 478.12 GB, percent = 25.4%
[2023-11-17 11:04:42,840] [INFO] [stage3.py:479:_setup_for_real_optimizer] optimizer state initialized

here are the code snippets and config file

       model = AutoModelForCausalLM.from_pretrained(
        model_config["path"], 
        config=auto_config, 
        device_map="auto" if device is None else device,
        offload_folder="./data_cache_dir",
        torch_dtype=torch.bfloat16 if use_bf16 else torch.float16, 
        trust_remote_code=True)

   AdamOptimizer = DeepSpeedCPUAdam if args.offload else FusedAdam
    optimizer = AdamOptimizer(model.parameters(),lr=args.learning_rate if args.max_lr is None else args.max_lr, betas=(0.9, 0.95))

    num_update_steps_per_epoch = math.ceil(len(train_dataloader) / args.gradient_accumulation_steps)
    total_steps = args.num_train_epochs * num_update_steps_per_epoch
    lr_scheduler = get_scheduler(
        name=args.lr_scheduler_type,
        optimizer=optimizer,
        num_warmup_steps=args.num_warmup_steps,
        num_training_steps=total_steps
    )
    requires_grad_parameter = filter(lambda p : p.requires_grad, model.parameters())
    see_memory_usage("memory info Before deepspeed initialize hf-model")
    model_engine, optimizer, _, lr_scheduler = deepspeed.initialize(
        model=model,
        optimizer=optimizer,
        model_parameters=requires_grad_parameter,
        args=args,
        config=ds_config,
        lr_scheduler=lr_scheduler,
        dist_init_required=True)
    see_memory_usage("memory info Before deepspeed initialize hf-model")

{
  "train_batch_size": "auto",
  "train_micro_batch_size_per_gpu": "auto",
  "steps_per_print": 10,
  "zero_optimization": {
    "stage": 3,
    "offload_optimizer": {
      "device": "cpu",
      "pin_memory": true
    },
    "offload_param": {
      "device": "cpu",
      "pin_memory": true
    },
    "allgather_partitions": true,
    "allgather_bucket_size": 2e8,
    "reduce_scatter": true,
    "reduce_bucket_size": 2e8,
    "overlap_comm": true,
    "contiguous_gradients": true,
    "stage3_param_persistence_threshold": 1e4,
    "stage3_max_live_parameters": 3e7,
    "stage3_prefetch_bucket_size": 3e7,
    "memory_efficient_linear": false
  },
  "gradient_clipping": 0.8,
  "prescale_gradients": false,
  "bf16": {
    "enabled": true
  },
  "wall_clock_breakdown": false
}

looking forward you reply, thank you

Start at the DeepSpeed stage 3 initialization path shown by stage3.py:479 and compare its memory reporting with torch.cuda.memory_reserved() during deepspeed.initialize. Determine whether the reported reserved memory is expected for this configuration and document a reproducible explanation or required correction.