kvcache-ai / kvcache-ai/Mooncake
[Installation]: Build release wheels in the PyTorch manylinux image instead of ubuntu-22.04
- Dominant language
- C++
- Stars
- 6.6k
- Forks
- 1.2k
- Avg merge
- 3d 5h
- Merged PRs (30d)
- 312
Description
## Issue
The release wheels are built on a plain `ubuntu-22.04` runner (`release.yaml`, `release-cuda13.yaml`, `pre-release.yaml`, etc.). I'd like to build them inside the PyTorch manylinux images instead, the way PyTorch and vLLM build their wheels for broad system support.
## Background
Building on ubuntu-22.04 ties the wheels to that environment in two ways:
- **glibc floor of 2.35.** `build_wheel.sh` tags the wheel from the runner's glibc, so it comes out `manylinux_2_35`. That leaves out RHEL/Rocky/Alma 9 (glibc 2.34), the RHEL 8 family (2.28), Ubuntu 20.04 (2.31), and a lot of HPC images.
- **The toolchain's libstdc++ leaks in.** ubuntu-22.04 ships GCC 12, so the binaries require `GLIBCXX_3.4.30`. glibc and libstdc++ are versioned separately, so a host can satisfy the `manylinux_2_35` floor and still fail to load:
```
mooncake_master: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.30' not found
```
`import mooncake.store` fails the same way. #2585 statically links the C++ runtime to stop that leak, but the wheel still carries the 2.35 floor.
## Proposal
Build in `pytorch/manylinux2_28-builder` (AlmaLinux 8, glibc 2.28) — the standard image for CUDA Python wheels:
- `:cuda12.8` and `:cuda13.0` tags cover both CUDA lines; matching aarch64 images exist for ARM.
- The CUDA toolkit is already in the image, so the `Jimver/cuda-toolkit` step goes away.
- Python lives at `/opt/python/cp${XY}-cp${XY}/`, and `dependencies.sh` already has a `yum` path for AlmaLinux.
- `build_wheel.sh` already runs `auditwheel repair`; on a 2.28 base it just tags `manylinux_2_28`.
vLLM did exactly this with its `BUILD_OS=manylinux` build base (`pytorch/manylinux2_28-builder:cuda13.0`) to get a glibc 2.28 floor that matches PyTorch's own wheels - see `vllm/docker/Dockerfile` and https://github.com/vllm-project/vllm/pull/41416.
### Before submitting a new issue...
- [x] Make sure you already searched for relevant issues and read the [documentation](https://kvcache-ai.github.io/Mooncake/)
Contributor guide
Research direction
Start by comparing the wheel jobs in release.yaml, release-cuda13.yaml, and pre-release.yaml, then read build_wheel.sh and dependencies.sh for their current runner and dependency assumptions. Check the PyTorch manylinux2_28-builder image and its CUDA/Python paths. Done means the release workflows build and repair wheels there, using the bundled CUDA toolkit and producing the intended manylinux_2_28 artifacts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, docker, github-actions, python
- Domain
- build-system, ci-cd, devops
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 65/100