[BUG] init_inference loads qwen3-32b model very slow but train model loads it quickly

I try to use deepspeed.init_inference to load qwen3-32b model for inference in 4 L20(80GB memory each). It costs about 1 hour to finish the init_inference.
But when I use the huggingface transformer's trainer api with deepspeed enable, it only costs about 10min to load the model to the gpu and start training.

I add some log to figure out which method cost too much time and found it's the below code
https://github.com/deepspeedai/DeepSpeed/blob/f394e7803611f31157c5f449ba0c7f01859e936a/deepspeed/inference/engine.py#L172

that makes me very confused.

I also found the gpu memory usage increase very slow when use init_inference.
It increase only aboud 800M every 5s.

Start at deepspeed/inference/engine.py around line 172, where the report observes the slow initialization, and compare that loading path with the Hugging Face Trainer path using DeepSpeed. Reproduce the timing and GPU-memory progression for Qwen3-32B on four L20 GPUs; done means the cause is identified and a verified resolution or limitation is documented.