kvcache-ai / kvcache-ai/Mooncake
[Bug] RDMA transport fails cross-subnet: missing route resolution for gateway MAC
- Dominant language
- C++
- Stars
- 6.6k
- Forks
- 1.2k
- Avg merge
- 3d 5h
- Merged PRs (30d)
- 312
Description
## Bug Report: RDMA Transport Cross-Subnet Failure — Missing Route Resolution
### Summary
Mooncake's RDMA transport (both the original `RdmaEndPoint` and TENT `RdmaEndPoint`) fails with `transport retry counter exceeded` when peers are on different `/28` subnets. Same-subnet communication works correctly. Same NIC index across subnets fails.
### Root Cause
Mooncake exclusively uses **manual QP creation** (`ibv_create_qp` + `ibv_modify_qp`) with **no RDMA CM** involvement. The AH (Address Handle) attributes are populated with the peer's GID and LID directly, but **no gateway MAC address is resolved** for the Ethernet layer:
```cpp
// rdma_endpoint.cpp, INIT -> RTR transition
attr.ah_attr.grh.dgid = peer_gid; // peer GID directly
attr.ah_attr.grh.sgid_index = local_gid_index;
attr.ah_attr.grh.hop_limit = kMaxHopLimit; // hardcoded 16
attr.ah_attr.dlid = peer_lid; // peer LID directly
attr.ah_attr.is_global = 1;
attr.ah_attr.port_num = context_.portNum();
```
Within the same subnet, ARP automatically resolves the neighbor MAC, so `dgid` / `dlid` pointing to the peer works. Across subnets, the destination MAC must be the **gateway's MAC** (not the peer's), and the GID in the GRH carries the final destination — similar to IP routing where the Ethernet frame targets the next-hop router while the IP packet carries the ultimate destination.
The current code has **no**:
- Subnet prefix comparison (local vs. peer GID `subnet_prefix`)
- Gateway MAC / LID resolution
- Route resolver module
- Perftest-style `bind_source_ip` equivalent
`MC_RDMA_BIND_ADDRESS` only overrides the RPC server identity for dual-NIC environments — it does **not** perform source IP binding or route resolution.
### Affected Components
| Component | File | Status |
|-----------|------|--------|
| Original RdmaEndPoint | `mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp` | Affected |
| TENT RdmaEndPoint | `mooncake-transfer-engine/tent/src/transport/rdma/endpoint.cpp` | Affected |
| IBGDA DeviceTransport | `mooncake-transfer-engine/src/transport/device/ibgda_device_transport.cpp` | Affected |
### Proposed Fix
Add a `route_resolver` module that borrows RDMA CM's route resolution capability (equivalent to perftest's `--bind_source_ip`), but **does not** use RDMA CM to establish connections. The approach:
1. **Detect cross-subnet**: Compare `local_gid.global.subnet_prefix` with `peer_gid.global.subnet_prefix`
2. **Resolve route**: Use `rdma_resolve_addr()` + `rdma_resolve_route()` on a temporary `rdma_cm_id` to let the kernel resolve the path (gateway MAC, hop count, etc.), then extract the resolved AH attributes
3. **Apply resolved attributes**: Use the resolved gateway MAC / dlid in the AH attributes while keeping the peer GID in the GRH `dgid`
4. **Keep manual QP lifecycle**: Continue using `ibv_create_qp` + `ibv_modify_qp` for actual data-path QPs — RDMA CM is only used for route discovery
This mirrors the approach used by `ib_write_bw` (perftest) with `--bind_source_ip`, where RDMA CM resolves the route but the actual data transfer uses manually created QPs.
### Reproduction
1. Set up two hosts with RDMA NICs on different `/28` subnets (e.g., 10.0.1.x and 10.0.2.x), connected via a router
2. Configure Mooncake transfer engine between the two hosts
3. Observe: same-subnet transfers succeed, cross-subnet transfers fail with `transport retry counter exceeded`
### Environment
- RDMA NICs: Mellanox ConnectX-5/6 (RoCEv2)
- Network: Two `/28` subnets with a router in between
- Mooncake: latest main branch
Contributor guide
Assessment
This issue has not been assessed yet.