kvcache-ai / kvcache-ai/Mooncake
[Installation] Publish prebuilt ROCm/HIP wheels to PyPI
- Dominant language
- C++
- Stars
- 6.6k
- Forks
- 1.2k
- Avg merge
- 3d 5h
- Merged PRs (30d)
- 312
Description
## Summary
We are bringing up **MiniMax-M3 MXFP4 vLLM disaggregated benchmarks on AMD MI355X** with Mooncake DRAM KV offload (`MooncakeStoreConnector` + MultiConnector with MoRIIO P/D). HIP support landed in [#1742](https://github.com/kvcache-ai/Mooncake/pull/1742), but today the only PyPI packages are CUDA (`mooncake-transfer-engine`, `mooncake-transfer-engine-cuda13`), CPU-only (`mooncake-transfer-engine-non-cuda`), and NPU (`mooncake-transfer-engine-npu`). **There is no ROCm/HIP wheel.**
Our current workaround is to **build Mooncake from source at job runtime** with `-DUSE_HIP=ON -DUSE_CUDA=OFF` and cache the artifact — workable but slow on first run and hard to satisfy downstream CI policies that require pinned images to run as shipped (no runtime builds).
## Request
Please publish a **prebuilt ROCm/HIP Python wheel** to PyPI, e.g.:
- `mooncake-transfer-engine-rocm` (or `-hip`), versioned alongside existing releases (we pin `v0.3.11.post1` today)
Minimum requirements for our MI355X / ROCm 7.x stack:
- Built with `-DUSE_HIP=ON`, `-DWITH_STORE=ON`
- Includes Transfer Engine + Mooncake Store Python bindings (`mooncake.engine`, `mooncake.store`, `mooncake_master` CLI)
- Links against `libamdhip64.so` but **does not vendor incompatible ROCm runtime libs** (same approach validated in #1742 `dist-rocm-hip` wheels)
- `manylinux_2_28_x86_64` for cp310–cp313 (matching existing CUDA wheel matrix)
## Why not `mooncake-transfer-engine-non-cuda`?
The non-CUDA wheel is CPU-only. Our vLLM disagg path validates HIP linkage (`libamdhip64.so`) for the Store/TE stack on MI355X. CPU-only wheels do not satisfy this.
## Current workaround
Runtime build in `server_vllm.sh`:
```bash
cmake -S Mooncake -B build -DUSE_CUDA=OFF -DUSE_HIP=ON -DWITH_STORE=ON ...
Contributor guide
Assessment
This issue has not been assessed yet.