[BUG] MOQ Compression not work with ZeRO

Describe the bug
Scritpt：
https://github.com/microsoft/DeepSpeedExamples/blob/master/compression/gpt2/bash_script/run_zero_quant.sh
Config： https://github.com/microsoft/DeepSpeedExamples/blob/master/compression/gpt2/config/ds_config_W4or8A8_Qgroup64_fp16.json

In order to quantize larger models，I have to enable ZeRO，but unexpected behavior was found.

ZerO Enable + single GPU:
I have try ZeRO 1/2/3，however, the accuracy before and after quantization is exactly the same, indicating that MoQ was not working

ZerO Enable + muti-GPUs:
ds.init_inference() raises an expection that grad_acc is invalid (grad_acc = 0, which should be > 0)

To Reproduce
Edit the config
（https://github.com/microsoft/DeepSpeedExamples/blob/master/compression/gpt2/config/ds_config_W4or8A8_Qgroup64_fp16.json） and use it in run_zero_quant.sh
set stage of zero_optimization to be 1,2 or 3

for ZerO Enable + single GPU : just run run_zero_quant.sh again

for ZerO Enable + muti-GPUs : modify "nproc_per_node" to the number of GPUs in run_zero_quant.sh then run again

Expected behavior
ZerO Enable + single GPU: obtain similar quantization model accuracy with ZerO disable + single GPU
Zero Enable + muti-GPUs: obtain similar quantization model accuracy with ZerO disable + single GPU

ds_report output

Screenshots
If applicable, add screenshots to help expln your problem.

System info (please complete the following information):

• OS: Ubuntu 18.04.6 LTS
• GPU count and types: 3090 * 8
• Interconnects: 1G Eth
• Python version: 3.10
• [e.g. two machines with x8 A100s each] ny other relevant info ab (if applicable) [e.g., two machines connected with 100 Gbps IB] setup

Launcherxt
Are you launching your experiment with the deepspeed launcher, MPI, or something else?

Docker context
Are you using a specific docker image that you can share?

Additional context
Add any other context about the problem here.

Start with compression/gpt2/bash_script/run_zero_quant.sh and compression/gpt2/config/ds_config_W4or8A8_Qgroup64_fp16.json in DeepSpeedExamples, then trace the ds.init_inference() entry point for ZeRO stages 1, 2, and 3. Reproduce on one and multiple GPUs and compare quantized accuracy with ZeRO disabled. Done means MoQ changes accuracy as expected and multi-GPU initialization no longer reports grad_acc=0.