[BUG]zero2/3 stage training nccl error for deepspeed version > 0.9.5

while I am using the deepspeed package:
the following deepspeed configuration works well for deepspeed==0.9.5
{
"zero_optimization": {
"stage": 2,
"overlap_comm": true,
"contiguous_gradients": true,
"sub_group_size": 1e9,
"reduce_bucket_size": "auto"
},
"fp16": {
"enabled": true,
"auto_cast": false,
"loss_scale": 0,
"initial_scale_power": 16,
"loss_scale_window": 1000,
"hysteresis": 2,
"min_loss_scale": 1e-100
},
"train_batch_size": "auto",
"train_micro_batch_size_per_gpu": "auto",
"gradient_accumulation_steps": "auto",
"wall_clock_breakdown": false
}

But for deepspeed version > 0.9.5 (I test 0.12.6 and 0.13.1), NCCL error will happen:
      RuntimeError: NCCL error in: ../torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:1269, internal error, NCCL version 2.17.1
       File ".local/lib/python3.8/site-packages/transformers/trainer.py", line 1537, in train
            return inner_training_loop(
       File ".local/lib/python3.8/site-packages/transformers/trainer.py", line 1914, in _inner_training_loop
            self._maybe_log_save_evaluate(tr_loss, model, trial, epoch, ignore_keys_for_eval)
       File "local/lib/python3.8/site-packages/transformers/trainer.py", line 2242, in _maybe_log_save_evaluate
            tr_loss_scalar = self._nested_gather(tr_loss).mean().item()
       File "local/lib/python3.8/site-packages/transformers/trainer.py", line 3345, in _nested_gather
            tensors = distributed_concat(tensors)
       File ".local/lib/python3.8/site-packages/transformers/trainer_pt_utils.py", line 204, in distributed_concat
            dist.all_gather(output_tensors, tensor)
       File "/opt/conda/envs/ptca/lib/python3.8/site-packages/torch/distributed/distributed_c10d.py", line 2275, in all_gather
            work = default_pg.allgather([tensor_list], [tensor])
      RuntimeError: NCCL error in: ../torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:1269, internal error, NCCL version 2.17.1
      ncclInternalError: Internal check failed.

Do we have some breaking change for later version deepspeed package?