model save/load error using deepspeed zero3 with accelerate [model params size[0]]

Errors occur when I save models or load models using accelerate with deepspeed zero3 config.
There is no error when I use deepspeed zero2 or zero1.

The accelerate config yaml:

compute_environment: LOCAL_MACHINE
deepspeed_config:
 deepspeed_config_file: ds_config.json
 zero3_init_flag: true
distributed_type: DEEPSPEED
fsdp_config: {}
main_process_ip: null
main_process_port: null
main_training_function: main
use_cpu: false
machine_rank: 0
num_machines: 1
num_processes: 2

ds_config.json:

{
    "bf16": {
        "enabled": true
    },
    "zero_optimization": {
        "stage": 3,
        "offload_optimizer": {
            "device": "cpu",
            "pin_memory": true
        },
        "offload_param": {
            "device": "cpu",
            "pin_memory": true
        },
        "overlap_comm": true,
        "contiguous_gradients": true,
        "sub_group_size": 1e9,
        "stage3_max_live_parameters": 1e9,
        "stage3_max_reuse_distance": 1e9,
        "stage3_gather_16bit_weights_on_model_save": true
    },
    "activation_checkpointing": {
        "enabled": false
    },
    "gradient_accumulation_steps": 1,
    "steps_per_print": 1,
    "wall_clock_breakdown": false
}

ERROR1: saving model error

when I run:

  unwrapped_model = accelerator.unwrap_model(model)
  unwrapped_model.save_pretrained(
      save_path / "unwrapped_model",
      is_main_process=accelerator.is_main_process,
      save_function=accelerator.save,
      state_dict=accelerator.get_state_dict(model),
      safe_serialization=False
  )

I get a "*.bin" model file whose size is about only 100KB. It is definitely not the true model state file.

I finally save the model successfully using:

accelerator.save_model(model, str(save_path / "unwrapped_model"), safe_serialization=False)

I manually check the "pytorch_model.bin" file and ensure that the above code saves the model params successfully.

ERROR2: loading model error

When I run:

model = Qwen3()
state_dict = torch.load('pytorch_model.bin', map_location="cpu")
model.load_state_dict(state_dict, strict=False)

Errors occur:

...
size mismatch for backbone.model.model.layers.7.cross_attn.q_norm.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([0]).
size mismatch for backbone.model.model.layers.7.cross_attn.k_norm.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([0]).
size mismatch for backbone.model.model.layers.8.self_attn.q_proj.weight: copying a param with shape torch.Size([2048, 1024]) from checkpoint, the shape in current model is torch.Size([0]).
size mismatch for backbone.model.model.layers.8.self_attn.k_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([0]).
size mismatch for backbone.model.model.layers.8.self_attn.v_proj.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([0]).
size mismatch for backbone.model.model.layers.8.self_attn.o_proj.weight: copying a param with shape torch.Size([1024, 2048]) from checkpoint, the shape in current model is torch.Size([0]).
size mismatch for backbone.model.model.layers.8.self_attn.q_norm.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([0]).
size mismatch for backbone.model.model.layers.8.self_attn.k_norm.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([0]).
size mismatch for backbone.model.model.layers.8.mlp.gate_proj.weight: copying a param with shape torch.Size([3072, 1024]) from checkpoint, the shape in current model is torch.Size([0]).
size mismatch for backbone.model.model.layers.8.mlp.up_proj.weight: copying a param with shape torch.Size([3072, 1024]) from checkpoint, the shape in current model is torch.Size([0]).
size mismatch for backbone.model.model.layers.8.mlp.down_proj.weight: copying a param with shape torch.Size([1024, 3072]) from checkpoint, the shape in current model is torch.Size([0]).
size mismatch for backbone.model.model.layers.8.input_layernorm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([0]).
...

The error is likely the same as issues3574. But I do not find any solutions to it.

Reproduce the failure with the shown accelerate YAML and ds_config.json, comparing accelerator.get_state_dict(), accelerator.save_model(), and the Qwen3 load_state_dict path. Read DeepSpeed issue 3574 alongside the reported ZeRO-3 behavior, then verify that saving produces complete parameters and loading does not create zero-sized tensors.