kvcache-ai / kvcache-ai/Mooncake

[RFC]: Add Intel XPU support to Transfer Engine & Mooncake Store

Open
#4,031 1 comment 1 reaction 0 assignees View on GitHub
RFC
Dominant language
C++
Stars
6.6k
Forks
1.2k
Avg merge
3d 5h
Merged PRs (30d)
312

Description

## Background
Mooncake currently has **no Intel GPU (XPU) support** — there are no SYCL / oneAPI / `USE_XPU`
references in the tree. This issue tracks adding Intel Data Center GPU (Max / Arc) as a
first-class memory type, mirroring the existing CUDA / TPU backends.

Approach: **correctness first** (KV staged through host DRAM — works anywhere, low risk),
then **performance** (GPUDirect dma-buf RDMA, PCIe peer-to-peer). Each PR is independently
mergeable and lands with a test that validates it.

## Work breakdown

### PR1 — XPU platform + build enablement · **P0** · deps: none
- Add `USE_XPU` / `USE_TENT` build gate; oneAPI (icpx/SYCL) toolchain + Docker image.
- New `XpuPlatform`: `MTYPE_XPU`, `sycl::malloc_device` / `queue.memcpy`, pointer
classification via `sycl::get_pointer_type`.
- **Acceptance:** builds with `-DUSE_XPU=ON`; gtest device `alloc→copy→free` roundtrip;
`classify() == MTYPE_XPU` for device USM.

### PR2 — Transport, routing & topology · **P0** · deps: PR1
- `XpuTransport` capabilities (`gpu_to_dram` / `dram_to_gpu`); parse `"xpu:N"`;
`isGpuType` + `is_gpu`; host-staging policy.
- Device topology probe; robust USM device/shared/host + interior-pointer classification.
- **Acceptance:** unit tests — `getTypeEnum("xpu:0") == XPU`, both capability predicates
agree, staging selects D2H; mocked topology + device/shared/host/interior-pointer cases.

### PR3 — Store staging + Python e2e (MVP) · **P0** · deps: PR2
- Build-guarded VRAM→host staging in the mooncake-store client; `registerLocalMemory("xpu:0")`.
- pybind wheel + `MooncakeStore` XPU config; vLLM-XPU end-to-end path.
- **Acceptance:** store integration `Put`/`Get` byte-equal via host staging; pytest wheel
import; vLLM-XPU smoke `prefill→store→decode` returns tokens.

### PR4 — GPUDirect dma-buf RDMA · **P1** · deps: PR2
- Export a dma-buf FD from a SYCL USM allocation (Level Zero interop); `ibv_reg_dmabuf_mr`;
`RdmaTransport` advertises `gpu_*` for direct VRAM RDMA, with staged fallback.
- **Acceptance:** on a dma-buf-capable node, VRAM→remote-VRAM RDMA is correct with 0 host
copies; falls back to staging when unsupported.

### PR5 — PCIe intra-node P2P · **P2** · deps: PR4
- `device.ext_oneapi_can_access_peer` + enable peer access; direct device-to-device copy
over PCIe.
- **Acceptance:** on a 2-XPU node, `can_access_peer` → PCIe P2P copy is correct;
bandwidth > host-staged.

### PR6 — Benchmarks, docs & CI · **P1** · deps: PR3, PR4
- tebench XPU backend + multi-NIC; TTFT / throughput vs the CUDA baseline; build/deploy
docs; CI e2e job on an Intel GPU runner.
- **Acceptance:** tebench emits TTFT/throughput and regression-checks against a recorded
baseline; CI e2e smoke passes on the Intel GPU runner.

## Dependency graph
```
PR1 → PR2 → { PR3, PR4 }
PR4 → PR5
{ PR3, PR4 } → PR6
```

Contributor guide

Open the contributing guide

Research direction

Start with the existing CUDA and TPU backends, then inspect the build configuration and the Transfer Engine and mooncake-store components named in the RFC. Begin with PR1: add the XPU build gate and XpuPlatform, and run the gtest alloc→copy→free and pointer-classification checks. Done for that slice means a -DUSE_XPU=ON build and passing device-USM tests; later PRs require their listed transport, store, RDMA, P2P, and CI acceptance tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, docker, python
Domain
backend, distributed-systems, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.