kvcache-ai / kvcache-ai/Mooncake

[TE][CXI]: Split transfers that straddle BufferDesc chunk boundaries

Open
#3,502 1 comment 0 reactions 0 assignees View on GitHub
feature request Transfer Engine
Dominant language
C++
Stars
6.6k
Forks
1.2k
Avg merge
3d 5h
Merged PRs (30d)
312

Description

Parent: #3500

## Problem

`CxiTransport::registerLocalMemory` already auto-splits buffers larger than `max_mr_size` into multiple `BufferDesc`s.

`CxiTransport::submitTransferTask` still posts **one slice per request**. A transfer that crosses a chunk seam fails `selectDevice`.

Until this is fixed, Store must keep splitting CXI segments in `GetTransportRegistrationLimit()`.

## Proposed work

Same as EFA / RDMA: cap each posted slice at source and target `BufferDesc` remainder, then bind keys per sub-slice.

## Test plan

Needs CXI / Slingshot hardware. Suggested coverage matching `rdma_large_mr_test.cpp` (destination seam, source seam, past-first-chunk WRITE).

I do not have CXI hardware to implement or verify this.

Contributor guide

Open the contributing guide

Research direction

Start with CxiTransport::registerLocalMemory and CxiTransport::submitTransferTask, then compare the proposed slicing behavior with the EFA/RDMA implementation. Review rdma_large_mr_test.cpp for destination-seam, source-seam, and past-first-chunk WRITE coverage. Done means each posted slice respects both BufferDesc remainders and binds keys per sub-slice, with CXI hardware tests passing.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.