[REQUEST] Model serving via deepspeed's inference module

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
I am trying to run my model serving code in a model-parallel fashion. The tutorial shows how to run code on multi-GPU but the data is predefined, which cannot be used for serving. My original code is using fastapi to do the serving work. When using deepspeed --num_gpus n example.py the fastapi server will also be initiated n times, which cause port conflict.

Describe alternatives you've considered
Do I have to first start the model in parallel using deepspeed in one script and then start another script for fastapi, and finally connect them somehow?

Additional context
None.

Start with the multi-GPU tutorial and the model-serving code using FastAPI mentioned in the issue. Determine the intended DeepSpeed inference entry point and how server initialization should work without port conflicts across processes. Done should mean a documented or supported model-serving approach for this setup.