Training GRPO on multi nodes of NPU and rollout on other node of GPU, but calls port 51216 of server insetead of vllm_server_port of server

Describe the bug

Train grpo while rollout called port 51216 of server host, but got timeout.
Port 51216 is the default VLLMClient group_ports, it seems that vllm_server_port 9883 was not passed.

• 1 node (GPU-A100) for rollout
  CUDA_VISIBLE_DEVICES=1
  swift rollout \
  --model $MODEL_PATH
  --port 9883
  --vllm_max_model_len 32768
  --vllm_gpu_memory_utilization 0.9

• 2 nodes(NPU-910B) for train

• while rollout server has started up, run curl http://172.19.12.153:9883/health/ on training nodes, it returns {"status": "ok"}

• error info

Your hardware and system info
Write your system info like CUDA version/system/GPU/torch version here(在这里给出硬件信息和系统信息，如CUDA版本，系统，GPU型号和torch版本等)

• GPU-A100 of 1 node for rollout
  CUDA12.4.1
  ms_swift==3.10.3
  torch==2.6.0
  vllm==0.8.5

• NPU-910B of 2 nodes for train
  CANN8.1.RC1
  ms_swift==3.10.3
  torch==2.7.1
  torch_npu==2.7.1
  deepspeed==0.17.6

Additional context
Add any other context about the problem here(在这里补充其他信息)

Start by tracing how the GRPO rollout client receives and propagates vllm_server_port, especially where the default VLLMClient group port 51216 is selected. Reproduce the setup with the provided swift rollout --port 9883 command and health check; done means training requests reach port 9883 instead of timing out on 51216.