[BUG] misc false： parameter object has no attribute "comm"

    return self._return_mics_optimizer(optimizer, timers)
  File "/home/ma-user/anaconda3/envs/PyTorch-1.8/lib/python3.7/site-packages/deepspeed/runtime/engine.py", line 1525, in _return_mics_optimizer
    communication_data_type=self.communication_data_type)
  File "/home/ma-user/anaconda3/envs/PyTorch-1.8/lib/python3.7/site-packages/deepspeed/runtime/zero/mics.py", line 396, in __init__
    self.dp_process_group = first_param.comm.param_shard_group
AttributeError: 'Parameter' object has no attribute 'comm'

I use the ds_config as follows:

    "zero_optimization": {
        "stage": 3,
        "offload_optimizer": {
            "device": "cpu",
            "pin_memory": true
        },
        "offload_param": {
            "device": "cpu",
            "pin_memory": true
        },
        "mics_shard_size": 4,
        "mics_hierarchical_params_gather": false,
        "overlap_comm": true,
        "contiguous_gradients": true,
        "stage3_max_live_parameters" : 1e9,
        "stage3_max_reuse_distance" : 1e9,
        "stage3_prefetch_bucket_size" : 5e8,
        "stage3_param_persistence_threshold" : 1e6,
        "sub_group_size" : 1e12,
        "stage3_gather_16bit_weights_on_model_save": true

what i know : i know i need to call misc_init instead of deepspeed init after I view the deepspeed.py code at 378 line.

i need to know: how and where can I call the misc_init ?

Bests

Start with deepspeed/runtime/engine.py around _configure_zero_optimizer and _return_mics_optimizer, then inspect deepspeed/runtime/zero/mics.py around its constructor and the misc_init path mentioned near deepspeed.py line 378. Reproduce with the supplied ZeRO-3 configuration and determine the supported initialization sequence. Done means the configuration initializes without the missing comm attribute and the required call path is clear.