mudler / mudler/LocalAI

Feature request: official gfx1151 (Strix Halo / RDNA3.5) backend builds on ROCm 7.14

Open
#10,883 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/backends area/ci enhancement
Dominant language
Go
Stars
49.2k
Forks
4.5k
Avg merge
1d 3m
Merged PRs (30d)
239

Description

Is your feature request related to a problem? Please describe.

Users on gfx1151 (AMD Ryzen AI Max+ / Strix Halo, RDNA3.5) must build the LocalAI backends (llama-cpp, diffusers, rerankers, whisper) themselves on every ROCm/LocalAI bump. The official backend gallery ships CUDA and some ROCm variants, but no gfx1151 target. Until ROCm 7.13 gfx1151 was preview-only / not in AMD's official catalog, so official builds weren't justifiable. ROCm 7.14 (2026-07-15) makes gfx1151 a first-class, natively-detected target — the historical blocker is gone, and Strix Halo (96–128 GB unified memory, strong MoE decode) is a fast-growing local-inference platform.

Describe the solution you'd like

Add gfx1151 / RDNA3.5 on ROCm 7.14 as an official target in the backend build matrix, published to the gallery like other backends. We run a production gfx1151 stack and are happy to (1) contribute a PR wiring the target into the build workflow using the recipe below, and (2) act as a gfx1151 CI/validation partner — we have the hardware to smoke/perf-test backend releases.

Verified build recipe (production, Ryzen AI Max+ 395):

  • Base userspace — Ubuntu 24.04 + two AMD metapackages from the multi-arch repo:
    deb https://repo.amd.com/rocm/packages-multi-arch/ubuntu2404 stable main
    amdrocm7.14-gfx1151            # runtime closure (blas/rand/fft/solver/sparse, ck, dnn,
                                   # rccl/rocshmem, math-common=rocWMMA, amdsmi/hipify/decode/jpeg)
    amdrocm-core-sdk7.14-gfx1151   # build closure (core-dev + developer-tools + rdc + opencl)
    
    Two metapackages pull the complete coherent 7.14 set — no transitive gaps (structurally fixes the librocprofiler-register.so class of missing-lib crashes).
  • llama.cpp build flags: -DGGML_HIP=ON -DGGML_HIP_ROCWMMA_FATTN=ON -DAMDGPU_TARGETS=gfx1151 plus -DGGML_HIP_NO_VMM=ON (HIP VMM is unreliable on gfx1151).
  • Runtime: HSA_OVERRIDE_GFX_VERSION is not needed on 7.14 (native detection; it was only a 7.12/7.13 workaround). For large single buffers (>~61 GiB) the kernel needs ttm.page_pool_size/ttm.pages_limit raised (the "64 GB cliff", ggml-org/llama.cpp#15018) — worth a docs note for Strix Halo users running big MoE (we run up to ~83 GB models at a 120 GiB GTT ceiling).
  • Shared-lib packaging: derive the runtime lib allowlist dynamically (recursive ldd closure) rather than a static list — transitive ROCm deps shift per version (librocprofiler-register.so, libamd_comgr_loader.so, …).

Describe alternatives you've considered

  • Self-building (our current approach): works, but is the single most expensive recurring maintenance item — a custom base image plus atomic llama-cpp/diffusers/rerankers/whisper rebuilds on every ROCm/LocalAI sync.
  • Community-only build guide: helps individuals but doesn't give gfx1151 users the "pull from gallery" experience the other targets have.

Additional context

Would this be something you'd want in-tree, and if so, in what shape — a build-matrix entry, a separate ROCm/gfx1151 workflow, or docs + a community build guide? Happy to align on scope first, then open the PR. We can validate on real gfx1151 hardware for each backend release.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate the backend build matrix and gallery-publishing workflow, then compare existing CUDA and ROCm target entries with the requested gfx1151 recipe. Confirm scope for all four backends, ROCm 7.14 packaging, dynamic runtime-library closure, and the large-buffer documentation note, then validate releases on gfx1151 hardware.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, ubuntu
Domain
ai, backend, build-system
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.