[REQUEST] Can I specify certain parameters' grads and optimizer states not being partitioned in stage 3?

Is your feature request related to a problem? Please describe.
I can exclude certain small parameters from being partitioned by setting the 'persistence_threshold' config. However, it seems that their gradients and optimizer states are still partitioned in stage 3. Given that existing PEFT methods, like LoRA, have already achieved relatively low memory demands for both gradients and optimizer states, I'm wondering if there's a solution to keep their gradients and optimizer states intact.

Describe the solution you'd like
Due to the cost of grads and optim states are also related to the params' size, can deepspeed also refrain the grads and optim states of specified params from being partitioned?

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Start by tracing DeepSpeed ZeRO stage 3 handling of the persistence_threshold configuration and how it partitions gradients and optimizer states. Compare the behavior for persisted parameters with the requested LoRA or PEFT use case. Done means a supported way to specify parameters whose gradients and optimizer states remain unpartitioned, with validation for stage 3 behavior.