Question: recommended method for inference from a pipelined model checkpoint?

Is it possible to load a checkpoint that was saved by a pipelined model back into DeepSpeed for inference, i.e., with deepspeed.init_inference?

In particular, I have trained a model using 2-way tensor and 4-way pipeline parallelism so that I have a set of layer checkpoint files like:

global_step26000/layer_01-model_00-model_states.pt
global_step26000/layer_01-model_01-model_states.pt
<snip>
global_step26000/layer_44-model_00-model_states.pt
global_step26000/layer_44-model_01-model_states.pt

I would now like to load this checkpoint to do inference, ideally with deepspeed.init_inference for optimized performance. I've tried a number of different approaches, but I haven't quite cracked it.

Is there a way to do that, or can it only be loaded back with deepspeed.initialize?

Thanks.

Start with the checkpoint layout under global_step26000 and the deepspeed.init_inference and deepspeed.initialize entry points mentioned in the question. Determine whether the pipelined checkpoint can be loaded for inference, and document or implement a clear supported loading path if one exists.