kvcache-ai / kvcache-ai/Mooncake
[RFC]: Add Intel XPU support to Transfer Engine & Mooncake Store
- Dominant language
- C++
- Stars
- 6.6k
- Forks
- 1.2k
- Avg merge
- 3d 5h
- Merged PRs (30d)
- 312
Description
## Background
Mooncake currently has **no Intel GPU (XPU) support** — there are no SYCL / oneAPI / `USE_XPU`
references in the tree. This issue tracks adding Intel Data Center GPU (Max / Arc) as a
first-class memory type, mirroring the existing CUDA / TPU backends.
Approach: **correctness first** (KV staged through host DRAM — works anywhere, low risk),
then **performance** (GPUDirect dma-buf RDMA, PCIe peer-to-peer). Each PR is independently
mergeable and lands with a test that validates it.
## Work breakdown
### PR1 — XPU platform + build enablement · **P0** · deps: none
- Add `USE_XPU` / `USE_TENT` build gate; oneAPI (icpx/SYCL) toolchain + Docker image.
- New `XpuPlatform`: `MTYPE_XPU`, `sycl::malloc_device` / `queue.memcpy`, pointer
classification via `sycl::get_pointer_type`.
- **Acceptance:** builds with `-DUSE_XPU=ON`; gtest device `alloc→copy→free` roundtrip;
`classify() == MTYPE_XPU` for device USM.
### PR2 — Transport, routing & topology · **P0** · deps: PR1
- `XpuTransport` capabilities (`gpu_to_dram` / `dram_to_gpu`); parse `"xpu:N"`;
`isGpuType` + `is_gpu`; host-staging policy.
- Device topology probe; robust USM device/shared/host + interior-pointer classification.
- **Acceptance:** unit tests — `getTypeEnum("xpu:0") == XPU`, both capability predicates
agree, staging selects D2H; mocked topology + device/shared/host/interior-pointer cases.
### PR3 — Store staging + Python e2e (MVP) · **P0** · deps: PR2
- Build-guarded VRAM→host staging in the mooncake-store client; `registerLocalMemory("xpu:0")`.
- pybind wheel + `MooncakeStore` XPU config; vLLM-XPU end-to-end path.
- **Acceptance:** store integration `Put`/`Get` byte-equal via host staging; pytest wheel
import; vLLM-XPU smoke `prefill→store→decode` returns tokens.
### PR4 — GPUDirect dma-buf RDMA · **P1** · deps: PR2
- Export a dma-buf FD from a SYCL USM allocation (Level Zero interop); `ibv_reg_dmabuf_mr`;
`RdmaTransport` advertises `gpu_*` for direct VRAM RDMA, with staged fallback.
- **Acceptance:** on a dma-buf-capable node, VRAM→remote-VRAM RDMA is correct with 0 host
copies; falls back to staging when unsupported.
### PR5 — PCIe intra-node P2P · **P2** · deps: PR4
- `device.ext_oneapi_can_access_peer` + enable peer access; direct device-to-device copy
over PCIe.
- **Acceptance:** on a 2-XPU node, `can_access_peer` → PCIe P2P copy is correct;
bandwidth > host-staged.
### PR6 — Benchmarks, docs & CI · **P1** · deps: PR3, PR4
- tebench XPU backend + multi-NIC; TTFT / throughput vs the CUDA baseline; build/deploy
docs; CI e2e job on an Intel GPU runner.
- **Acceptance:** tebench emits TTFT/throughput and regression-checks against a recorded
baseline; CI e2e smoke passes on the Intel GPU runner.
## Dependency graph
```
PR1 → PR2 → { PR3, PR4 }
PR4 → PR5
{ PR3, PR4 } → PR6
```
Contributor guide
Research direction
Start with the existing CUDA and TPU backends, then inspect the build configuration and the Transfer Engine and mooncake-store components named in the RFC. Begin with PR1: add the XPU build gate and XpuPlatform, and run the gtest alloc→copy→free and pointer-classification checks. Done for that slice means a -DUSE_XPU=ON build and passing device-USM tests; later PRs require their listed transport, store, RDMA, P2P, and CI acceptance tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, docker, python
- Domain
- backend, distributed-systems, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100