mudler / mudler/vllm.cpp

Evaluate native HRX and Loom on gfx1100 with matched measurements

Open
#3,080 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: BACKEND-ROCM

Evaluate native AMD HRX and Loom for the project on gfx1100. The developer requested hardware measurements before selecting an architecture. Existing vllm.cpp code is a comparator, not a preferred design or correctness oracle.

Build pinned HRX and the author-linked llama.cpp HRX integration. Establish native gfx1100 execution and correctness, compare HRX and conventional HIP on identical supported workloads, separate runtime-submission and kernel/compiler contributions, and assess vllm.cpp integration including Qwen3.5 GDN and quantized arms. Native HRX, its HIP compatibility layer, and selective Loom reuse are all candidates.

Starting anchors: ROCm/hrx-system 6bcd5a4ff111fa5bf160ab9f4592ca8e7cc810b1; AMD-Ecosystem/llama.cpp hrx-graph-develop-v2 at 6319038132ed12f968ea68f37753f705da830ea8; https://github.com/ggml-org/llama.cpp/discussions/27219. Establish exact-pair build compatibility. Spec: .agents/specs/rocm-hrx-evaluation.md, to be committed before implementation.

Record revisions, commands, binary/model hashes, device identity and contention, output correctness, cold-start/JIT and warm prefill/decode/latency/memory. Product correctness uses the applicable pinned oracle and unchanged thresholds. Upstream benchmark or compiler-smoke results are diagnostic evidence, not end-to-end vllm.cpp acceptance.

Finish with an independently reviewed empirical recommendation: adopt, pursue a specific measurable gap, or reject for measured reasons. Missing coverage is engineering scope to investigate. Shipped defaults require the normal acceptance gates.

The BACKEND-ROCM operator owns this evaluation. Task staging and builds stay in ignored build directories, with no /tmp or /dev/shm workspaces. Preserve compact reproducible evidence and remove disposable artifacts when no longer needed.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with .agents/specs/rocm-hrx-evaluation.md and the pinned HRX and llama.cpp revisions, then establish the exact-pair builds and native gfx1100 execution. Record reproducible hashes, commands, correctness, startup, warm performance, latency, memory, and contention across the specified workloads. Done means an independently reviewed recommendation to adopt, pursue a measured gap, or reject the approaches.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, machine-learning, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.