kvcache-ai / kvcache-ai/Mooncake
[RFC]: TENT capability-aware routing and RDMA latency direct path
- Dominant language
- C++
- Stars
- 6.6k
- Forks
- 1.2k
- Avg merge
- 3d 5h
- Merged PRs (30d)
- 312
Description
## Changes proposed
This issue tracks the four stacked PRs that add capability-aware path selection and latency-oriented direct dispatch to the TENT runtime.
The current TENT dispatch path needs a clearer separation between:
- what memory locations and memory types each transport can actually reach;
- how a request is mapped to a direct or staged transfer path;
- how intent/deadline information affects path cost and runtime priority;
- how an RDMA latency-sensitive request can bypass the scheduled worker queue when a safe direct path is available.
The stack below implements those pieces incrementally so each PR has a focused review surface.
## PR stack
1. #3728: Add capability graph path synthesis
Adds `CapabilityPathSynthesizer` and transport capability declarations. The runtime can now synthesize direct or staged paths from source/target memory types, local/remote locations, staging candidates, and per-transport reachability.
Main effect: route selection becomes capability-driven instead of being encoded as transport-specific ad hoc checks in the submit path.
2. #3729: Remove the legacy transport resolve path
Removes the old `resolveTransport` style dispatch path once the capability graph route resolver exists. Submit and resubmit now use the same resolved route representation, including transport type, device mask, QP pool, and optional staging parameters.
Main effect: there is one route resolution path to reason about before adding intent-aware scoring.
3. #3730: Add intent-aware direct path policy
Adds `DirectPathPolicy`, request priority normalization from intent/deadline, and path scoring inputs such as runtime queue pressure and RDMA load state. This PR also extends capability graph tests and adds direct path policy tests.
Main effect: latency-sensitive requests, throughput-oriented requests, and explicit direct-path mode are represented as policy decisions instead of scattered conditionals.
This PR intentionally does not add the RDMA direct-post runtime implementation.
4. #3731: Add RDMA latency-optimized direct posting
Adds the RDMA runtime support for direct posting after the policy layer exists. This includes direct CQ/QP setup, direct lane ownership per context, direct completion polling from status/cancel paths, direct slice accounting, and endpoint lookup shared by scheduled and direct paths.
The direct path checks `RdmaEndPoint::isDirectReady()` before posting. If the peer has not negotiated `direct_qp_num`, the direct path declines and the request falls back to the scheduled RDMA path.
Main effect: eligible latency-sensitive RDMA requests can use a direct posting path, while unsupported or non-ready direct paths fall back safely.
## Review focus
Reviewers should evaluate the stack as a TENT runtime routing/direct-dispatch change, not as a classic RDMA transport worker ownership refactor.
Important points to check:
- capability graph path synthesis selects the expected direct or staged path;
- all submit/resubmit paths consume the same resolved route state after the legacy resolver is removed;
- intent/deadline priority normalization does not regress existing explicit priority handling;
- direct path policy only attempts direct dispatch for the intended request classes or explicit force mode;
- RDMA direct posting declines cleanly when the direct QP is unavailable or not negotiated;
- scheduled RDMA remains the fallback path for unsupported direct dispatch cases.
## Local validation
Focused TENT tests were built and run at the stack head:
```bash
cmake -S . -B build
cmake --build build --target tent_capability_graph_test direct_path_policy_test tent_transport_selector_test tent_runtime_queue_dispatch_test tent_rail_monitor_test tent_qos_contract_test tent_topology_priority_matrix_test -j16
ctest --test-dir build -R '^(tent_capability_graph_test|direct_path_policy_test|tent_transport_selector_test|tent_runtime_queue_dispatch_test|tent_rail_monitor_test|tent_qos_contract_test|tent_topology_priority_matrix_test)$' --output-on-failure
```
Result: all 7 tests passed.
`pre-commit` was not available in the local environment (`pre-commit: command not found`), so PR-scoped pre-commit could not be run locally.
Contributor guide
Research direction
Read PRs #3728 through #3731 as the implementation map, then review the listed capability graph, direct path policy, transport selector, and runtime queue dispatch tests. Run the provided CMake build and focused ctest command to verify the seven named tests. Done means the stacked routing and RDMA direct-posting changes are reviewed, with safe fallback to scheduled RDMA preserved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- distributed-systems, networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 20/100