[REQUEST] Ability to switch out weights when using deepspeed inference

Is your feature request related to a problem? Please describe.
If I have multiple fine-tuned versions of the same base model where only a small number of weights are different, I'd like to be able to change out the weights quickly without relaunching deepspeed. Not sure if this is already possible, so marking this issue as a feature request.

Describe the solution you'd like
Have a straightforward and fast way to reload weights to specific modules within a pytorch model, that has already been prepared by deepspeed for multi-gpu single-node inference, across multiple processes.

Describe alternatives you've considered
If not using deepspeed for inference, it is quite simple to reload weights for a small portion of the model.

Additional context
I'm wondering if this is possible for models for which deepspeed inference does not support custom kernels (T5), and models that deepspeed inference does support custom kernels for (GPT)

Start by reviewing DeepSpeed inference and the PyTorch model preparation path for multi-GPU, single-node, multi-process execution. Determine how weights for specific modules could be reloaded without relaunching, including models with and without DeepSpeed custom kernels. Done means a defined, fast reload mechanism works across processes for both cases.