Test hangs when world_size=4 and reuse_dist_env=True on PyTorch >= 2.7.0

When the

• PyTorch version is greater than 2.7.0,
• world_size=4
• reuse_dist_env=True,

The test will get stuck at tests/unit/common.py ->_dist_destroy(self) ->dist.destroy_process_group()

reproduce file:

from unit.common import DistributedTest
import pytest
@pytest.mark.parametrize("tp_size", [2, 4])
class TestEmpty(DistributedTest):
    world_size = 4
    reuse_dist_env = True **# wsz=4 and reuse_dist_env=True will hang.**

    def test(self, tp_size: int):
        print("finished test")
        return 

We can temporarily avoid this issue by setting reuse_dist_env=False

Start with the reproduction using PyTorch greater than 2.7.0, world_size=4, and reuse_dist_env=True. Read tests/unit/common.py, especially DistributedTest._dist_destroy and its call to dist.destroy_process_group(). Compare the behavior with reuse_dist_env=False and define done as the parametrized test completing without hanging.