How zero3 releases the GPU memory occupied by parameters？

I would like to know how zero3 releases the GPU memory occupied by parameters after forward ends. From the code, it seems that _post_forward_module_hook will call release_sub_module to partition param and release param by param.data = torch.empty(0). Param.data is indeed modified to 0. However, I found that GPU memory did not decrease. When I release the GPU memory occupied by parameters through register_full_backward_hook after backward ends with the same operation, the GPU memory does decrease. Is this due to reference counting? I would like to know how deepspeed solves this problem. Thank you very much.

Start with the _post_forward_module_hook and release_sub_module entry points mentioned in the issue, then compare their parameter handling with the reported register_full_backward_hook path. Trace references around param.data = torch.empty(0) and document the observed difference in GPU memory release; done means a clear explanation backed by the relevant DeepSpeed behavior.