[REQUEST] provide utility function to consolidate the optimizer state

Is your feature request related to a problem? Please describe.
Yes. Two scenarios

• During a training with deepspeed, the job may fail with fp16 (zero=1). Then, I have to resort to fp32 to continue the training (zero=0). If I stick to deepspeed (with zero=0), the restore utility cannot restore teh state from the previous checkpoint (from zero=1). If the feature is implemented, I can restore the state of the optimizer and use non-deepspeed code to continue the training.
• Sometimes, we have fewer resource and use, e.g. 64 GPUs to do the training. After, say, 1 week, we have more, e.g. 256GPUs. Then, I need to continue the training. All here are with deepspeed. If the feature is implemented, then the optimizer can be restored to the previous status before initilizing with deepspeed, although deepspeed might have a dedicated code path for this scenarios.

Describe the solution you'd like
A function to receive the checkpoint folder as input and output the consolidated optimizer state.

Describe alternatives you've considered
No.

No file, test, or entry point is named. Start by reviewing the existing checkpoint restore utility and the optimizer paths for DeepSpeed ZeRO stages 0 and 1; done means a checkpoint folder can produce a consolidated optimizer state usable before initializing a different DeepSpeed configuration.