[BUG]run oom with load optim

**Describe the bug**
if run with --no-load-optim, it is ok, memory usage is about 60% on H20. But with load optim, it is oom

**Stack trace/logs**
TENCENT64:47604:53083 [0] include/alloc.h:103 NCCL WARN Cuda failure 2 'out of memory'
TENCENT64:47604:53083 [0] include/alloc.h:206 NCCL WARN Failed to CUDA calloc async 24 bytes
TENCENT64:47611:53081 [7] include/alloc.h:103 NCCL WARN Cuda failure 2 'out of memory'
TENCENT64:47611:53081 [7] include/alloc.h:206 NCCL WARN Failed to CUDA calloc async 4 bytes
RuntimeError: CUDA error: CUBLAS_STATUS_ALLOC_FAILED when calling `cublasCreate(handle)`

**Environment (please complete the following information):**
- Megatron-LM 0.11
- PyTorch 2.5.1
- NCCL version 2.21.5+cuda12.4