lablup / lablup/mlxcel

Support Windows and Linux x86_64 binary builds and release artifacts

Open
#25 0 comments 0 reactions 1 assignee Claimed by @achimnol View on GitHub
priority:medium status:ready type:enhancement
Dominant language
Rust
Stars
467
Forks
54
Avg merge
4h 25m
Merged PRs (30d)
310

Description

## Summary

Extend mlxcel's release build matrix to cover Linux x86_64 and Windows in addition to the current macOS aarch64 + Linux aarch64 (CUDA) targets. This aligns the distribution surface with the Backend.AI:GO desktop application and makes mlxcel deployable to customer sites that standardize on Linux x86_64 (the dominant CUDA host architecture in production).

## Background

Today's releases ship two artifact families:

- `mlxcel-macos-aarch64.zip` — Apple Silicon
- `mlxcel-linux-aarch64-cuda13-{gb10,gh200}` — Linux aarch64 + CUDA (Grace Blackwell / Grace Hopper)

The README prerequisites already claim "Linux (aarch64 or x86_64)" support, but no published binary exists for x86_64 today. Windows is not built at all, although the runtime distribution layout (MLX upstream's `mlx/backend/cuda/delayload.cpp` delay-load mechanism + NVIDIA DLL bundling, ~1.0–1.3 GB total) has been worked out.

_[Update 2026-06-25: the Linux x86_64 + CUDA artifact now ships (since v0.3.0, PR #262). Windows remains unbuilt and is tracked in detail by #58.]_

Two motivations:

1. **Customer fit** — Most enterprise CUDA hosts (RTX, A100, L40, H100/H200 outside of GH200 Grace systems) are Linux x86_64. Without a published x86_64 binary, those sites cannot adopt mlxcel without building from source.
2. **Backend.AI:GO parity** — The Backend.AI:GO desktop application supports macOS + Windows + Linux x86_64. mlxcel should match so the same runtime can serve any Backend.AI:GO target.

## Proposed Solution

Split the work into three deliverables.

### A. Linux x86_64 + CUDA build

> **Status (2026-06-25): Done (PR #262).** The `mlxcel-linux-x86_64-cuda13.zip` CLI + server artifacts ship from v0.3.0 onward (present in latest v0.3.3). CI runs a 10-token GPU smoke test on an RTX 5060 (sm_120) self-hosted runner. The CUDA arch list is `80;86;89;90a;100;120` (Ampere through Blackwell), broader than the proposal below.

- Add a release artifact `mlxcel-linux-x86_64-cuda13.{tar.gz|zip}` produced from an x86_64 + NVIDIA host (build-only is acceptable for the first iteration; smoke test on a real GPU is preferred).
- Decide on the target CUDA architectures — SM 80/86/89/90 covers Ampere through Hopper; the existing CUDA arch matrix in `README.md` already documents non-Hopper quantization limitations.
- Verify `src/lib/mlxcel-core/build.rs` and the bundled MLX C++ build produce equivalent output on x86_64 — expected to be mechanical because Linux paths already differentiate from macOS via `#[cfg(target_os = "linux")]`.

### B. Windows + CUDA build

> **Status (2026-06-25): Open, tracked in detail by #58** (native Windows + CUDA feasibility spike + porting plan). The bullets below map to #58's Layer 2 (`build.rs` Windows branches, static-link delay-load replication) and Phases 2-5. Upstream picture per #58: MLX now builds and CI-tests Windows CPU, and the CUDA backend ships Windows delay-load plumbing (including the static-library case), but there is no Windows+CUDA CI, so it is plumbed yet unproven.

- Extend `src/lib/mlxcel-core/build.rs` with `#[cfg(target_os = "windows")]` branches:
- MSVC toolchain detection (vs. GCC/Clang on Linux)
- `CUDA_PATH` env var (Windows convention) in addition to `CUDA_HOME`
- Link against `cudart.lib` and friends; static-vs-dynamic linkage decision mirrors MLX's Windows build
- Add Windows handling in the bundled MLX C++ build for OpenBLAS FetchContent and the delay-load DLL compile definitions (`MLX_CUDA_BIN_DIR`, `MLX_CUDNN_BIN_DIR`).
- Package the Windows artifact (`mlxcel.exe` + `nvidia/cublas/bin/...` etc.) per the runtime distribution layout. Bundle size is expected to be ~1.0–1.3 GB because of cuBLAS + NVRTC + cuDNN — fits within the 2 GB-per-asset GitHub Release limit.
- Code signing — investigate `signtool.exe` / Azure Code Signing, or ship unsigned for the first release with a documented SmartScreen warning.
- Confirm all crates compile on `x86_64-pc-windows-msvc`: `tokenizers`, `safetensors`, `axum`, `cxx`, `sentencepiece-sys`, and the multimodal stack (ffmpeg / video frame extraction may need a Windows-friendly variant).

### C. Documentation + README updates

> **Status (2026-06-25): Partial.** CUDA arch coverage and the Linux aarch64/x86_64 differentiation are documented in `docs/installation.md` (Supported-platforms table `:13-18`, release section `:120-122`, arch selection `:97-122`). The Windows build guide and the README Windows enumeration remain pending #58.

- Update `README.md` prerequisites to enumerate Linux aarch64, Linux x86_64, and Windows distinctly rather than the current "Linux (aarch64 or x86_64)" shorthand.
- Add a `docs/windows-build-guide.md` for developers building from source on Windows.
- Update the CUDA architecture compatibility table to reflect the broader x86_64 GPU coverage (RTX 30/40/50 series, A100, L40, H100).

## Implementation Notes

- **Code surface that already touches Windows**
- `src/distributed/rdma_capabilities.rs` already has `cfg!(target_os = "windows")` branches. The rest of `src/distributed/` should be audited for Windows-specific socket / transport quirks before claiming pipeline parallelism works on Windows.
- No other `target_os = "windows"` gates exist today. (Confirmed 2026-06-25; #58's Layer 3 audit measures ~79 Unix-API sites total, 39 in `src/distributed/`, with all hard primitives io_uring/kqueue/RDMA inside the distributed module.)
- **MLX upstream Windows status** — MLX has the delay-load CUDA mechanism (`mlx/backend/cuda/delayload.cpp`), but full Windows-only correctness may require additional upstream patches. _[Update 2026-06-25: see #58; upstream MLX now CI-tests Windows CPU and ships CUDA delay-load plumbing including the static-library case, but has no Windows+CUDA CI yet.]_
- **Self-hosted runner needs** — x86_64 + NVIDIA Linux runner (RTX class is enough for build, but actual smoke testing benefits from Hopper or Blackwell). Windows + NVIDIA runner ideally has the same.
- **Build-only vs. test-on-platform** — For the first iteration, building plus a smoke test (model load + 10-token generate) on each platform is sufficient. Full benchmark + parity runs can land in a follow-up.
- **Rust target triples**
- Linux x86_64: `x86_64-unknown-linux-gnu`
- Windows: `x86_64-pc-windows-msvc`

The release pipeline lives in the development repository and not this mirror; this issue tracks the user-visible deliverable. Implementation discussion is welcome here.

_[Update 2026-06-25: the release workflow `.github/workflows/release.yml` runs in this repo and now produces the macOS, Linux aarch64, and Linux x86_64 CUDA artifacts (PR #262).]_

## Acceptance Criteria

- [x] Release publishes `mlxcel-linux-x86_64-cuda13.{tar.gz|zip}` (#262; ships from v0.3.0, present in v0.3.3 as `.zip` for both `mlxcel` and `mlxcel-server`, each with `.sha256`)
- [ ] Release publishes `mlxcel-windows-x86_64-cuda13.zip` with bundled CUDA runtime DLLs
- Tracked by #58; not yet built (no Windows job in `.github/workflows/release.yml`).
- [x] Smoke test (`mlxcel generate` with a 0.6B 4-bit model, 10 tokens) succeeds on Linux x86_64 + NVIDIA GPU (#262; CI step "Smoke test (10-token generate on GPU)" downloads `mlx-community/Qwen3-0.6B-4bit` and runs `mlxcel generate -n 10` on an RTX 5060 / sm_120 self-hosted runner)
- [ ] Smoke test succeeds on Windows + NVIDIA GPU
- Blocked on Deliverable B / #58.
- [ ] `README.md` prerequisites section enumerates Linux aarch64, Linux x86_64, and Windows distinctly
- Done: `docs/installation.md` differentiates Linux aarch64 (GH200/GB200/GB10) and x86_64 (Ampere through Blackwell) in the Supported-platforms table (`:13-18`) and release section (`:120-122`); `README.md:26` names Linux x86_64 + aarch64.
- Remaining: Windows is still listed only as "not documented here"; no distinct Windows prerequisites yet (pending #58).
- [ ] `docs/windows-build-guide.md` walks through a developer-side Windows build
- Not created; depends on #58 producing a working Windows build.
- [x] CUDA architecture compatibility table reflects x86_64 GPU coverage (Ampere, Ada, Hopper, Blackwell) (#262; `docs/installation.md:97-122` documents the x86_64 list `80;86;89;90a;100;120`)
- [ ] Code-signing strategy decided for the Windows binary (signed / unsigned with documented warning / deferred)
- Pending #58 packaging; macOS uses rcodesign today, Windows signtool / Azure Code Signing undecided.

---

## Original Suggestion

> Let's match the compatibility matrix with Backend.AI:GO desktop application, and also make it available to our customer sites which often use Linux x86-64 environments.

---
**Refresh log**

- 2026-06-25: Refreshed against current code and releases (issue created 2026-05-18).
- Deliverable A (Linux x86_64 + CUDA) is done: PR #262 added the `build-linux-x86_64-cuda` release job; `mlxcel-linux-x86_64-cuda13.zip` (CLI + server) ships from v0.3.0 through v0.3.3, and CI runs a 10-token GPU smoke test (`mlx-community/Qwen3-0.6B-4bit`) on an RTX 5060 (sm_120) runner. Checked off acceptance criteria 1, 3, and 7.
- Deliverable B (Windows + CUDA) remains open and is now tracked in detail by #58 (feasibility spike + porting plan); acceptance criteria 2, 4, 6, and 8 stay unchecked with pointers to #58.
- Deliverable C is partial: CUDA arch coverage and Linux aarch64/x86_64 differentiation are documented in `docs/installation.md`; the Windows build guide and the README Windows enumeration are still pending.
- Recorded the CUDA arch decision: x86_64 targets `80;86;89;90a;100;120` (Ampere through Blackwell), broader than the originally proposed SM 80/86/89/90.
- Marked stale claims with dated updates rather than deleting them: the x86_64 binary is now published, and the release workflow (`.github/workflows/release.yml`) runs in this repo.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.