kvcache-ai / kvcache-ai/Mooncake

[Installation]: Build release wheels in the PyTorch manylinux image instead of ubuntu-22.04

Open
#2,586 4 comments 1 reaction 0 assignees View on GitHub
Dominant language
C++
Stars
6.6k
Forks
1.2k
Avg merge
3d 5h
Merged PRs (30d)
312

Description

## Issue

The release wheels are built on a plain `ubuntu-22.04` runner (`release.yaml`, `release-cuda13.yaml`, `pre-release.yaml`, etc.). I'd like to build them inside the PyTorch manylinux images instead, the way PyTorch and vLLM build their wheels for broad system support.

## Background

Building on ubuntu-22.04 ties the wheels to that environment in two ways:

- **glibc floor of 2.35.** `build_wheel.sh` tags the wheel from the runner's glibc, so it comes out `manylinux_2_35`. That leaves out RHEL/Rocky/Alma 9 (glibc 2.34), the RHEL 8 family (2.28), Ubuntu 20.04 (2.31), and a lot of HPC images.

- **The toolchain's libstdc++ leaks in.** ubuntu-22.04 ships GCC 12, so the binaries require `GLIBCXX_3.4.30`. glibc and libstdc++ are versioned separately, so a host can satisfy the `manylinux_2_35` floor and still fail to load:

```
mooncake_master: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.30' not found
```

`import mooncake.store` fails the same way. #2585 statically links the C++ runtime to stop that leak, but the wheel still carries the 2.35 floor.

## Proposal

Build in `pytorch/manylinux2_28-builder` (AlmaLinux 8, glibc 2.28) — the standard image for CUDA Python wheels:

- `:cuda12.8` and `:cuda13.0` tags cover both CUDA lines; matching aarch64 images exist for ARM.
- The CUDA toolkit is already in the image, so the `Jimver/cuda-toolkit` step goes away.
- Python lives at `/opt/python/cp${XY}-cp${XY}/`, and `dependencies.sh` already has a `yum` path for AlmaLinux.
- `build_wheel.sh` already runs `auditwheel repair`; on a 2.28 base it just tags `manylinux_2_28`.

vLLM did exactly this with its `BUILD_OS=manylinux` build base (`pytorch/manylinux2_28-builder:cuda13.0`) to get a glibc 2.28 floor that matches PyTorch's own wheels - see `vllm/docker/Dockerfile` and https://github.com/vllm-project/vllm/pull/41416.

### Before submitting a new issue...

- [x] Make sure you already searched for relevant issues and read the [documentation](https://kvcache-ai.github.io/Mooncake/)

Contributor guide

Open the contributing guide

Research direction

Start by comparing the wheel jobs in release.yaml, release-cuda13.yaml, and pre-release.yaml, then read build_wheel.sh and dependencies.sh for their current runner and dependency assumptions. Check the PyTorch manylinux2_28-builder image and its CUDA/Python paths. Done means the release workflows build and repair wheels there, using the bundled CUDA toolkit and producing the intended manylinux_2_28 artifacts.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, docker, github-actions, python
Domain
build-system, ci-cd, devops
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.