Investigate SymmetricMemoryTest.test_subgroup hipErrorInvalidValue on MI300

# Summary

SymmetricMemoryTest.test_subgroup hipErrorInvalidValue is always failing on mi300

AI-generated investigation notes

## Observed failure

`SymmetricMemoryTest.test_subgroup` failed on a four-GPU MI300 runner in the scheduled `periodic-rocm-mi300` distributed workflow:

- Run: https://github.com/pytorch/pytorch/actions/runs/31954545174
- Job: https://github.com/pytorch/pytorch/actions/runs/31954545174/job/95186084362

The failure occurs during symmetric-memory rendezvous:

```text
distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_subgroup
...
torch.distributed._symmetric_memory.rendezvous
return _SymmetricMemory.rendezvous(tensor, group_name)
torch.AcceleratorError: CUDA error: invalid argument
Search for `hipErrorInvalidValue` in the ROCm HIP documentation.
```

This happened late in a long-running distributed shard. It is distinct from the unsupported NCCL2 reconfiguration failures and the workflow-wide runtime problem.

## Existing issue history

[#161612](https://github.com/pytorch/pytorch/issues/161612), `DISABLED test_subgroup (__main__.SymmetricMemoryTest)`, is closed. No currently open issue matching this MI300 `hipErrorInvalidValue` failure was found.

Only one recent occurrence was confirmed during this investigation, so the first step should be to establish whether the failure is deterministic, intermittent, or a recurrence of the previously disabled behavior.

## Suggested investigation

1. Reproduce `SymmetricMemoryTest.test_subgroup` on a four-GPU MI300 machine.
2. Determine which rendezvous argument is rejected by HIP and whether subgroup rank or device mapping is invalid.
3. Compare the current test and implementation with the state covered by #161612.
4. Check whether the behavior is specific to MI300, the installed ROCm version, or all RCCL/ROCm configurations.
5. Add a targeted fix and regression test if reproducible; otherwise document and narrowly quarantine the affected configuration rather than hiding unrelated symmetric-memory coverage.

cc @awgu @wanchaol @fegin @fduwjj @wz337 @wconstab @d4l3k @pragupta @msaroufim @dcci @aditvenk @weifengpy @kapilsh

Start with distributed/test_symmetric_memory.py and run SymmetricMemoryTest.test_subgroup on a four-GPU MI300 runner. Trace the symmetric-memory rendezvous entry point and compare the current behavior with issue #161612 to determine whether subgroup rank, device mapping, or another argument triggers hipErrorInvalidValue. Done means the failure is reproduced and fixed with regression coverage, or the affected configuration is narrowly documented or quarantined.

Written by the indexing model from the issue text.