kvcache-ai / kvcache-ai/Mooncake

[Installation] Publish prebuilt ROCm/HIP wheels to PyPI

Open
#3,108 8 comments 2 reactions 2 assignees Claimed by @amd-arozanov View on GitHub
Dominant language
C++
Stars
6.6k
Forks
1.2k
Avg merge
3d 5h
Merged PRs (30d)
312

Description

## Summary

We are bringing up **MiniMax-M3 MXFP4 vLLM disaggregated benchmarks on AMD MI355X** with Mooncake DRAM KV offload (`MooncakeStoreConnector` + MultiConnector with MoRIIO P/D). HIP support landed in [#1742](https://github.com/kvcache-ai/Mooncake/pull/1742), but today the only PyPI packages are CUDA (`mooncake-transfer-engine`, `mooncake-transfer-engine-cuda13`), CPU-only (`mooncake-transfer-engine-non-cuda`), and NPU (`mooncake-transfer-engine-npu`). **There is no ROCm/HIP wheel.**

Our current workaround is to **build Mooncake from source at job runtime** with `-DUSE_HIP=ON -DUSE_CUDA=OFF` and cache the artifact — workable but slow on first run and hard to satisfy downstream CI policies that require pinned images to run as shipped (no runtime builds).

## Request

Please publish a **prebuilt ROCm/HIP Python wheel** to PyPI, e.g.:

- `mooncake-transfer-engine-rocm` (or `-hip`), versioned alongside existing releases (we pin `v0.3.11.post1` today)

Minimum requirements for our MI355X / ROCm 7.x stack:

- Built with `-DUSE_HIP=ON`, `-DWITH_STORE=ON`
- Includes Transfer Engine + Mooncake Store Python bindings (`mooncake.engine`, `mooncake.store`, `mooncake_master` CLI)
- Links against `libamdhip64.so` but **does not vendor incompatible ROCm runtime libs** (same approach validated in #1742 `dist-rocm-hip` wheels)
- `manylinux_2_28_x86_64` for cp310–cp313 (matching existing CUDA wheel matrix)

## Why not `mooncake-transfer-engine-non-cuda`?

The non-CUDA wheel is CPU-only. Our vLLM disagg path validates HIP linkage (`libamdhip64.so`) for the Store/TE stack on MI355X. CPU-only wheels do not satisfy this.

## Current workaround

Runtime build in `server_vllm.sh`:

```bash
cmake -S Mooncake -B build -DUSE_CUDA=OFF -DUSE_HIP=ON -DWITH_STORE=ON ...

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.