kvcache-ai / kvcache-ai/Mooncake

[Bug] RDMA transport fails cross-subnet: missing route resolution for gateway MAC

Open
#2,468 3 comments 0 reactions 1 assignee Claimed by @zhangzhiqiangcs View on GitHub
Dominant language
C++
Stars
6.6k
Forks
1.2k
Avg merge
3d 5h
Merged PRs (30d)
312

Description

## Bug Report: RDMA Transport Cross-Subnet Failure — Missing Route Resolution

### Summary

Mooncake's RDMA transport (both the original `RdmaEndPoint` and TENT `RdmaEndPoint`) fails with `transport retry counter exceeded` when peers are on different `/28` subnets. Same-subnet communication works correctly. Same NIC index across subnets fails.

### Root Cause

Mooncake exclusively uses **manual QP creation** (`ibv_create_qp` + `ibv_modify_qp`) with **no RDMA CM** involvement. The AH (Address Handle) attributes are populated with the peer's GID and LID directly, but **no gateway MAC address is resolved** for the Ethernet layer:

```cpp
// rdma_endpoint.cpp, INIT -> RTR transition
attr.ah_attr.grh.dgid = peer_gid; // peer GID directly
attr.ah_attr.grh.sgid_index = local_gid_index;
attr.ah_attr.grh.hop_limit = kMaxHopLimit; // hardcoded 16
attr.ah_attr.dlid = peer_lid; // peer LID directly
attr.ah_attr.is_global = 1;
attr.ah_attr.port_num = context_.portNum();
```

Within the same subnet, ARP automatically resolves the neighbor MAC, so `dgid` / `dlid` pointing to the peer works. Across subnets, the destination MAC must be the **gateway's MAC** (not the peer's), and the GID in the GRH carries the final destination — similar to IP routing where the Ethernet frame targets the next-hop router while the IP packet carries the ultimate destination.

The current code has **no**:
- Subnet prefix comparison (local vs. peer GID `subnet_prefix`)
- Gateway MAC / LID resolution
- Route resolver module
- Perftest-style `bind_source_ip` equivalent

`MC_RDMA_BIND_ADDRESS` only overrides the RPC server identity for dual-NIC environments — it does **not** perform source IP binding or route resolution.

### Affected Components

| Component | File | Status |
|-----------|------|--------|
| Original RdmaEndPoint | `mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp` | Affected |
| TENT RdmaEndPoint | `mooncake-transfer-engine/tent/src/transport/rdma/endpoint.cpp` | Affected |
| IBGDA DeviceTransport | `mooncake-transfer-engine/src/transport/device/ibgda_device_transport.cpp` | Affected |

### Proposed Fix

Add a `route_resolver` module that borrows RDMA CM's route resolution capability (equivalent to perftest's `--bind_source_ip`), but **does not** use RDMA CM to establish connections. The approach:

1. **Detect cross-subnet**: Compare `local_gid.global.subnet_prefix` with `peer_gid.global.subnet_prefix`
2. **Resolve route**: Use `rdma_resolve_addr()` + `rdma_resolve_route()` on a temporary `rdma_cm_id` to let the kernel resolve the path (gateway MAC, hop count, etc.), then extract the resolved AH attributes
3. **Apply resolved attributes**: Use the resolved gateway MAC / dlid in the AH attributes while keeping the peer GID in the GRH `dgid`
4. **Keep manual QP lifecycle**: Continue using `ibv_create_qp` + `ibv_modify_qp` for actual data-path QPs — RDMA CM is only used for route discovery

This mirrors the approach used by `ib_write_bw` (perftest) with `--bind_source_ip`, where RDMA CM resolves the route but the actual data transfer uses manually created QPs.

### Reproduction

1. Set up two hosts with RDMA NICs on different `/28` subnets (e.g., 10.0.1.x and 10.0.2.x), connected via a router
2. Configure Mooncake transfer engine between the two hosts
3. Observe: same-subnet transfers succeed, cross-subnet transfers fail with `transport retry counter exceeded`

### Environment

- RDMA NICs: Mellanox ConnectX-5/6 (RoCEv2)
- Network: Two `/28` subnets with a router in between
- Mooncake: latest main branch

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.