Multi-node GRPO training fails on P5en.48xlarge (H200) due to Ray+NCCL issue

## Environment

- **Nodes**: 2x AWS P5en.48xlarge (8x NVIDIA H200 143GB per node, NVSwitch intra-node)
- **Networking**: 16 EFA v2 devices per node, EKS Kubernetes with `hostNetwork: true`
- **Image**: `public.ecr.aws/v9l4g5s4/nemo-rl-full:v2.1.0-p5en`
- **NCCL**: 2.27.5, aws-ofi-nccl 1.18.0, Libfabric 2.4

## Configuration

```yaml
cluster:
num_nodes: 2
num_gpus_per_node: 8

policy:
model_name: Qwen/Qwen2.5-1.5B
megatron_cfg:
enabled: true

# DTensor backend (no Megatron checkpoint conversion needed)
```

## Problem

Multi-node GRPO training completes successfully on P5.48xlarge (H100) but fails on P5en.48xlarge (H200) with an NCCL broadcast timeout during the `prepare_refit_info` phase (Megatron weight transfer).

### P5.48xlarge (H100) — WORKS

- 5/5 GRPO training steps completed successfully
- NCCL AllReduce and broadcast operations complete normally
- Full training run finishes without errors

### P5en.48xlarge (H200) — FAILS

- Training initializes correctly (Ray cluster forms, vLLM starts, Megatron sharding works)
- **Fails at**: `prepare_refit_info` — NCCL broadcast timeout when transferring weights from policy workers to vLLM workers
- Error: `torch.distributed.DistBackendError: NCCL communicator was aborted... ncclInternalError: Internal check failed` after 600s timeout

## Root cause

Ray's `CUDA_VISIBLE_DEVICES` remapping (each actor with `num_gpus=1` gets a remapped device index) breaks NCCL topology detection on H200 nodes with NVSwitch. This causes NCCL to misidentify the hardware topology and fall back to extremely slow transport.

**Evidence from standalone NCCL testing on the same P5en nodes:**

| Launcher | AllReduce 4MB | Notes |
|----------|---------------|-------|
| torchrun | 1.5 ms | Correct — uses NVSwitch + EFA |
| Ray actors | 3,660 ms (4KB) | 2400x slower, hangs on larger tensors |

The torchrun standalone test fully passes on P5en, confirming the hardware (GPUs, NVSwitch, EFA) is healthy. The issue is specifically in how Ray manages GPU assignment for NCCL on NVSwitch topology.

Filed upstream: ray-project/ray (NCCL AllReduce 2400x slower via Ray actors vs torchrun on H200)

## Fixes already applied successfully

These are NOT the issue — they were resolved prior to hitting the NCCL problem:

1. **Gloo loopback fix** (ref: #1121): Fixed `/etc/hosts` so hostname resolves to real node IP, not 127.0.0.1. Did NOT set `GLOO_SOCKET_IFNAME` globally (interface names differ across nodes: `enp72s0` vs `enp74s0`).
2. **`run_config.yaml` creation**: Auto-generated at `/opt/NeMo-RL/run_config.yaml` before training launch.
3. **EFA VPC CNI prefix delegation**: Configured AWS VPC CNI with `ENABLE_PREFIX_DELEGATION=true` to expose all 16 EFA devices per pod.
4. **NCCL_SOCKET_IFNAME**: Set to `^lo,docker,veth,eni` exclusion pattern (works across all node types).

## Suggestion

Investigate a **torchrun-based launcher** as an alternative to Ray for Megatron workers on H200/NVSwitch nodes. Since torchrun achieves correct NCCL performance on P5en, a torchrun-based backend for the distributed Megatron workers would bypass the Ray CUDA_VISIBLE_DEVICES remapping issue entirely.

Alternatively, if Ray must be used, consider:
- Preserving original `CUDA_VISIBLE_DEVICES` ordering and using `LOCAL_RANK` for GPU assignment instead of remapping
- Exposing all GPUs to each actor and using `torch.cuda.set_device()` explicitly

## Versions

- NeMo-RL: v2.1.0 (from image tag)
- Ray: 2.44.1
- PyTorch: 2.x
- NCCL: 2.27.5
- CUDA: 12.x