lablup / lablup/mlxcel

ci(rocm): self-hosted gfx1151 runner that builds, links and smoke-tests ROCm changes

Open
#1,811 0 comments 0 reactions 0 assignees View on GitHub
area:core platform:linux priority:medium status:backlog type:chore
Dominant language
Rust
Stars
467
Forks
54
Avg merge
4h 25m
Merged PRs (30d)
310

Description

Part of #1801. Phase 3. Depends on #1802.

## Context

No workflow has an AMD runner. Because the ROCm side of every MLX pin bump now lives in `src/lib/mlx-cpp/patches-rocm/`, a bump that breaks the ROCm build or runtime would go unnoticed without a job that links and runs it. The CUDA history shows the risk of green-but-hollow jobs: on the `81ba1c6a` bump, CUDA checks were green while nothing linked (`cargo check` does not link, the sm_70 job skipped on a CUDA 13 runner, and the xla-link job had no path trigger for pin changes).

A Linux host with a Radeon 8060S (`gfx1151`) is available as a self-hosted runner. It also runs other GPU services, so jobs must account for shared GPU memory and load.

## Scope

A ROCm CI job on a self-hosted runner that builds with `--features rocm`, links, and runs a short generation, triggered by every change that can affect the ROCm build.

## Implementation plan

1. Register the runner with a dedicated label (for example `rocm-gfx1151`) and document the host prerequisites (ROCm version, OpenBLAS/LAPACKE, Rust toolchain, disk for `target/` and `_deps/`).
2. Add a `rocm` job: `cargo build --release --features rocm`, then `mlxcel generate -n 10 --temp 0` on the `qwen3-0.6b-4bit` CI fixture from the public release `ci-fixtures/qwen3-0.6b-4bit-v1`, run with `LD_LIBRARY_PATH` unset, asserting a non-empty, non-garbage output and a GPU device in the diagnostics.
3. Add a `rocm` path filter in `.github/workflows/ci.yml` (filters are defined around lines 104-160) covering `src/lib/mlx-cpp/patches-rocm/**`, `src/lib/mlx-cpp/CMakeLists.txt`, `src/lib/mlxcel-core/build.rs`, `src/lib/mlxcel-core/build_support/**`, the bridge `cpp/` sources and `Cargo.toml` feature changes. The MLX pin line must trigger it.
4. Fail the job (not skip) if the runner is offline for a pin-bump PR, or make the skip visible in the PR checks.
5. Keep the job's GPU footprint small and document how it coexists with other GPU tenants on the host.

## Acceptance criteria

- [ ] A PR touching `patches-rocm/` or the MLX pin runs the ROCm job, which builds, links and generates.
- [ ] The job log shows the binary linked against the ROCm libraries and ran on the GPU.
- [ ] A deliberately broken overlay file fails the job.
- [ ] Unrelated PRs do not trigger it.

## References

- `.github/workflows/ci.yml` (path filters around 104-160)
- CI fixture: public release `ci-fixtures/qwen3-0.6b-4bit-v1`

Contributor guide

Open the contributing guide

Research direction

Start with .github/workflows/ci.yml, especially the path filters around lines 104-160, then review the listed ROCm sources and the public qwen3-0.6b-4bit-v1 CI fixture. Confirm the self-hosted gfx1151 prerequisites and shared-GPU constraints before implementing the job. Done means affected PRs build, link, run generation on the GPU, fail for a broken overlay, and unrelated PRs do not trigger it.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp, github-actions, rust
Domain
ci-cd, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.