[BUG]deepspeed zero3 gets error in dist.get_rank() in multiple node and multiple gpu

Describe the bug
deepspeed zero3 gets error in dist.get_rank() in multiple node and multiple gpu
it is perfectly fine when setting to stage 2

transformers: v.4.36.0
accelerate: v.0.26.0
deepspeed: v.0.12.6

To Reproduce
my configuration is
"zero_optimization": {
"stage": 3,
"offload_param": {
"device": "cpu",
"pin_memory": false
},
"offload_optimizer": {
"device": "cpu",
"pin_memory": true
},
"overlap_comm": true,
"contiguous_gradients": true,
"sub_group_size": 1e8,
"reduce_bucket_size": "auto",
"stage3_prefetch_bucket_size": "auto",
"stage3_param_persistence_threshold": "auto",
"stage3_max_live_parameters": 1e8,
"stage3_max_reuse_distance": 1e8,
"stage3_gather_16bit_weights_on_model_save": true
},
"gradient_clipping": "auto",
"steps_per_print": 2000,
"train_batch_size": "auto",
"wall_clock_breakdown": false,
"train_micro_batch_size_per_gpu": 1,
"gradient_accumulation_steps": 1

Steps to reproduce the behavior:

1. run the mixtral moe 8*7b full model with torchrun
2. it directly go to the function /home/tiger/.local/lib/python3.9/site-packages/deepspeed/runtime/config.py
   try:
   self.global_rank = dist.get_rank()
   if mpu is None:
   self.world_size = dist.get_world_size()
   else:
   self.world_size = mpu.get_data_parallel_world_size()
   except:
   self.global_rank = 0
   self.world_size = 1
3. remove the try except, the dist.get_rank() reports error. The same code is fine when stage is set to 2

Start at deepspeed/runtime/config.py and inspect the shown dist.get_rank() call in the configuration initialization path. Reproduce the Mixtral MoE 8x7B multi-node, multi-GPU run with torchrun using the supplied ZeRO-3 settings, and capture the actual exception before comparing it with ZeRO-2. Done means the ZeRO-3 failure is explained and the reproduction no longer fails under the reported setup.