mudler / mudler/vllm.cpp

Establish the current pinned vLLM oracle on Strix Halo before packed-decode optimization

Open
#3,043 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: BACKEND-GATE-ROCM-VLLM

The approved Strix Qwen3.8-27B optimization sequence starts by building and running vLLM at e126687a9a828d513c01a07cd69f025f27d63280. The installed oracle still reports 5559679229. Read-only leases 48587907-d9ac-47e8-8284-e13dbc8cc959, d24f3a86-70fc-4d77-b4c1-54deb998715b, and eab2c163-9fe2-44a5-b886-1dc33f9c9224 confirmed the old runtime and the pinned native GGUF plugin, not current-pin model gateability.

Owner: this Strix campaign operator, row BACKEND-GATE-ROCM-VLLM. Spec: .agents/specs/rocm-strix-vllm-headpin.md, committed before implementation.

Scope: isolated source build and generation on strix:gpu0 through rc, preserving the old installation. Pin vLLM and its GGUF plugin d4c1f0d082fc7cd4350da56689109a01c1f29d6c, prove source/archive/runtime/extension identity, run the identical Qwen3.8-27B Q4_K_M artifact and explicit prompt IDs, and retain output tokens, resolved config, environment, and failures. Capture the current upstream packed-recurrence tests before implementing the ROCm port. No global oracle-pin advance, no eager performance denominator, no inference-default changes or accepted throughput result while correctness fails.

Build in unique local storage with at most four jobs and ccache. Missing dependencies belong in isolated environments. A current-pin build failure remains a named failure, never justification to relabel the old pin. Diagnostic survey timings, if collected later, must carry TOKEN_GATE=FAIL until the declared gate actually passes.

Acceptance: reviewed reproducible harness with red/green and mutation evidence, operator build/run proof on the named device/model, committed source and runtime evidence. The subsequent packed-decode port is a separate issue/change and cannot use old CUDA fixtures as current-pin truth. Current upstream preserves FP32 beta (vllm#53877, 56058fd572), unlike the old donor. Related: #3015, #3040, #2921, #2534.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with .agents/specs/rocm-strix-vllm-headpin.md and verify the vLLM pin e126687a9a828d513c01a07cd69f025f27d63280 plus GGUF plugin pin d4c1f0d082fc7cd4350da56689109a01c1f29d6c. Build in unique local storage and run the Qwen3.8-27B Q4_K_M artifact on strix:gpu0, recording identity, configuration, environment, outputs, and failures. Done means reviewed red/green and mutation evidence with committed source and runtime proof, while preserving the old installation.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
build-system, machine-learning, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.