vllm-ascend server mode训练GRPO，init_communicator报错

ms-swift：3.7.3
vllm-ascend：v0.10.2rc1
tril：0.17.0到0.19.1都试了

swift rollout拉起成功，但运行grpo训练时，训练脚本中报错：

[INFO:swift] Successfully registered `/home/ma-user/work/ms-swift-gspo-vllm-infer/swift/llm/dataset/data/dataset_info.json`.
INFO 09-23 19:25:43 [__init__.py:36] Available plugins for group vllm.platform_plugins:
INFO 09-23 19:25:43 [__init__.py:38] - ascend -> vllm_ascend:register
INFO 09-23 19:25:43 [__init__.py:41] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load.
[INFO:swift] Setting args.remove_unused_columns: False
INFO 09-23 19:25:43 [__init__.py:207] Platform plugin ascend is activated
INFO 09-23 19:25:43 [__init__.py:36] Available plugins for group vllm.platform_plugins:
INFO 09-23 19:25:43 [__init__.py:38] - ascend -> vllm_ascend:register
INFO 09-23 19:25:43 [__init__.py:41] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load.
INFO 09-23 19:25:43 [__init__.py:36] Available plugins for group vllm.platform_plugins:
INFO 09-23 19:25:43 [__init__.py:38] - ascend -> vllm_ascend:register
INFO 09-23 19:25:43 [__init__.py:41] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load.
INFO 09-23 19:25:43 [__init__.py:207] Platform plugin ascend is activated
INFO 09-23 19:25:43 [__init__.py:207] Platform plugin ascend is activated
INFO 09-23 19:25:43 [__init__.py:36] Available plugins for group vllm.platform_plugins:
INFO 09-23 19:25:43 [__init__.py:38] - ascend -> vllm_ascend:register
INFO 09-23 19:25:43 [__init__.py:41] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load.
INFO 09-23 19:25:43 [__init__.py:207] Platform plugin ascend is activated
INFO 09-23 19:25:43 [__init__.py:36] Available plugins for group vllm.platform_plugins:
INFO 09-23 19:25:43 [__init__.py:38] - ascend -> vllm_ascend:register
INFO 09-23 19:25:43 [__init__.py:41] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load.
INFO 09-23 19:25:44 [__init__.py:36] Available plugins for group vllm.platform_plugins:
INFO 09-23 19:25:44 [__init__.py:38] - ascend -> vllm_ascend:register
INFO 09-23 19:25:44 [__init__.py:41] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load.
INFO 09-23 19:25:44 [__init__.py:207] Platform plugin ascend is activated
INFO 09-23 19:25:44 [__init__.py:207] Platform plugin ascend is activated
INFO 09-23 19:25:44 [__init__.py:36] Available plugins for group vllm.platform_plugins:
INFO 09-23 19:25:44 [__init__.py:38] - ascend -> vllm_ascend:register
INFO 09-23 19:25:44 [__init__.py:41] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load.
INFO 09-23 19:25:44 [__init__.py:207] Platform plugin ascend is activated
INFO 09-23 19:25:52 [utils.py:231] Found hccl from library libhccl.so
INFO 09-23 19:25:52 [pyhccl.py:83] vLLM is using pyhccl
======================
<class 'int'>
0
Traceback (most recent call last):
  File "/home/ma-user/work/ms-swift-gspo-vllm-infer/swift/cli/rlhf.py", line 5, in <module>
    rlhf_main()
  File "/home/ma-user/work/ms-swift-gspo-vllm-infer/swift/llm/train/rlhf.py", line 183, in rlhf_main
    return SwiftRLHF(args).main()
           ^^^^^^^^^^^^^^^
  File "/home/ma-user/work/ms-swift-gspo-vllm-infer/swift/llm/train/sft.py", line 27, in __init__
    super().__init__(args)
  File "/home/ma-user/work/ms-swift-gspo-vllm-infer/swift/llm/base.py", line 19, in __init__
    self.args = self._parse_args(args)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ma-user/work/ms-swift-gspo-vllm-infer/swift/llm/base.py", line 31, in _parse_args
    args, remaining_argv = parse_args(self.args_class, args)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ma-user/work/ms-swift-gspo-vllm-infer/swift/utils/utils.py", line 152, in parse_args
    args, remaining_args = parser.parse_args_into_dataclasses(argv, return_remaining_strings=True)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.13/lib/python3.11/site-packages/transformers/hf_argparser.py", line 358, in parse_args_into_dataclasses
    obj = dtype(**inputs)
          ^^^^^^^^^^^^^^^
  File "<string>", line 423, in __init__
  File "/home/ma-user/work/ms-swift-gspo-vllm-infer/swift/llm/argument/rlhf_args.py", line 130, in __post_init__
    self._init_external_vllm()
  File "/home/ma-user/work/ms-swift-gspo-vllm-infer/swift/llm/argument/rlhf_args.py", line 262, in _init_external_vllm
    self.vllm_client.init_communicator(device=get_current_device())
  File "/home/ma-user/work/ms-swift-gspo-vllm-infer/swift/trainers/rlhf_trainer/vllm_client.py", line 201, in init_communicator
    comm = PyNcclCommunicator(pg, device=0)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/vllm-workspace/vllm-ascend/vllm_ascend/distributed/device_communicators/pyhccl.py", line 113, in __init__
    self.unique_id = group.broadcast_obj(self.unique_id, src=0)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/vllm-workspace/vllm/vllm/distributed/utils.py", line 216, in broadcast_obj
    recv_obj = pickle.loads(self.store.get(key))
                            ^^^^^^^^^^^^^^^^^^^
torch.distributed.DistNetworkError: failed to recv, got 0 bytes

rollout服务中报错如下：

INFO:     127.0.0.1:60102 - "GET /health/ HTTP/1.1" 200 OK
INFO:     127.0.0.1:34444 - "GET /health/ HTTP/1.1" 200 OK
INFO:     127.0.0.1:41788 - "GET /get_world_size/ HTTP/1.1" 200 OK
INFO:     127.0.0.1:41788 - "POST /init_communicator/ HTTP/1.1" 200 OK
(EngineCore_DP0 pid=19473) INFO 09-23 19:25:52 [utils.py:231] Found hccl from library libhccl.so
(EngineCore_DP0 pid=19473) INFO 09-23 19:25:52 [pyhccl.py:83] vLLM is using pyhccl
(EngineCore_DP0 pid=19473) ======================
(EngineCore_DP0 pid=19473) <class 'NoneType'>
(EngineCore_DP0 pid=19473) None
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790] Invocation of collective_rpc method failed
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790] Traceback (most recent call last):
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 787, in _handle_client_request
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790]     result = method(*self._convert_msgspec_args(method, args))
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 423, in collective_rpc
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790]     return self.model_executor.collective_rpc(method, timeout, args,
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790]   File "/vllm-workspace/vllm/vllm/executor/uniproc_executor.py", line 58, in collective_rpc
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790]     answer = run_method(self.driver_worker, method, args, kwargs)
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790]   File "/vllm-workspace/vllm/vllm/utils/__init__.py", line 3060, in run_method
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790]     return func(*args, **kwargs)
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790]   File "/usr/local/python3.11.13/lib/python3.11/site-packages/trl/scripts/vllm_serve.py", line 108, in init_communicator
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790]     self.pynccl_comm = PyNcclCommunicator(pg, device=self.device)
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790]                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790]   File "/vllm-workspace/vllm-ascend/vllm_ascend/distributed/device_communicators/pyhccl.py", line 93, in __init__
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790]     assert isinstance(device, torch.device)
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=19473) ERROR 09-23 19:25:52 [core.py:790] AssertionError
INFO:     127.0.0.1:37672 - "GET /get_world_size/ HTTP/1.1" 200 OK