kvcache-ai / kvcache-ai/Mooncake

[RFC]: TENT capability-aware routing and RDMA latency direct path

Open
#3,733 1 comment 0 reactions 0 assignees View on GitHub
RFC
Dominant language
C++
Stars
6.6k
Forks
1.2k
Avg merge
3d 5h
Merged PRs (30d)
312

Description

## Changes proposed

This issue tracks the four stacked PRs that add capability-aware path selection and latency-oriented direct dispatch to the TENT runtime.

The current TENT dispatch path needs a clearer separation between:

- what memory locations and memory types each transport can actually reach;
- how a request is mapped to a direct or staged transfer path;
- how intent/deadline information affects path cost and runtime priority;
- how an RDMA latency-sensitive request can bypass the scheduled worker queue when a safe direct path is available.

The stack below implements those pieces incrementally so each PR has a focused review surface.

## PR stack

1. #3728: Add capability graph path synthesis

Adds `CapabilityPathSynthesizer` and transport capability declarations. The runtime can now synthesize direct or staged paths from source/target memory types, local/remote locations, staging candidates, and per-transport reachability.

Main effect: route selection becomes capability-driven instead of being encoded as transport-specific ad hoc checks in the submit path.

2. #3729: Remove the legacy transport resolve path

Removes the old `resolveTransport` style dispatch path once the capability graph route resolver exists. Submit and resubmit now use the same resolved route representation, including transport type, device mask, QP pool, and optional staging parameters.

Main effect: there is one route resolution path to reason about before adding intent-aware scoring.

3. #3730: Add intent-aware direct path policy

Adds `DirectPathPolicy`, request priority normalization from intent/deadline, and path scoring inputs such as runtime queue pressure and RDMA load state. This PR also extends capability graph tests and adds direct path policy tests.

Main effect: latency-sensitive requests, throughput-oriented requests, and explicit direct-path mode are represented as policy decisions instead of scattered conditionals.

This PR intentionally does not add the RDMA direct-post runtime implementation.

4. #3731: Add RDMA latency-optimized direct posting

Adds the RDMA runtime support for direct posting after the policy layer exists. This includes direct CQ/QP setup, direct lane ownership per context, direct completion polling from status/cancel paths, direct slice accounting, and endpoint lookup shared by scheduled and direct paths.

The direct path checks `RdmaEndPoint::isDirectReady()` before posting. If the peer has not negotiated `direct_qp_num`, the direct path declines and the request falls back to the scheduled RDMA path.

Main effect: eligible latency-sensitive RDMA requests can use a direct posting path, while unsupported or non-ready direct paths fall back safely.

## Review focus

Reviewers should evaluate the stack as a TENT runtime routing/direct-dispatch change, not as a classic RDMA transport worker ownership refactor.

Important points to check:

- capability graph path synthesis selects the expected direct or staged path;
- all submit/resubmit paths consume the same resolved route state after the legacy resolver is removed;
- intent/deadline priority normalization does not regress existing explicit priority handling;
- direct path policy only attempts direct dispatch for the intended request classes or explicit force mode;
- RDMA direct posting declines cleanly when the direct QP is unavailable or not negotiated;
- scheduled RDMA remains the fallback path for unsupported direct dispatch cases.

## Local validation

Focused TENT tests were built and run at the stack head:

```bash
cmake -S . -B build
cmake --build build --target tent_capability_graph_test direct_path_policy_test tent_transport_selector_test tent_runtime_queue_dispatch_test tent_rail_monitor_test tent_qos_contract_test tent_topology_priority_matrix_test -j16
ctest --test-dir build -R '^(tent_capability_graph_test|direct_path_policy_test|tent_transport_selector_test|tent_runtime_queue_dispatch_test|tent_rail_monitor_test|tent_qos_contract_test|tent_topology_priority_matrix_test)$' --output-on-failure
```

Result: all 7 tests passed.

`pre-commit` was not available in the local environment (`pre-commit: command not found`), so PR-scoped pre-commit could not be run locally.

Contributor guide

Open the contributing guide

Research direction

Read PRs #3728 through #3731 as the implementation map, then review the listed capability graph, direct path policy, transport selector, and runtime queue dispatch tests. Run the provided CMake build and focused ctest command to verify the seven named tests. Done means the stacked routing and RDMA direct-posting changes are reviewed, with safe fallback to scheduled RDMA preserved.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
distributed-systems, networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.