bench(BACKEND-GATE-ROCM-SGLANG): qualify Qwen3-4B BF16 for matched Strix comparison
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: BACKEND-GATE-ROCM-SGLANG
Owner: current Strix campaign operator. User approves Qwen/Qwen3-4B BF16 at 1cfa9a7208912126459214e8b04321603b3df60c for a matched four-engine Strix Halo benchmark: vllm.cpp, pinned vLLM, pinned SGLang, and stock llama.cpp b10451. Spec: .agents/specs/strix-four-engine-qwen3-4b.md (to be committed before implementation).
Build an isolated pinned SGLang ROCm environment for gfx1151; the existing CUDA/aarch64 recipe and MI300/MI350 Docker stages do not qualify. Use native safetensors for ours/vLLM/SGLang and audit a BF16 GGUF conversion for llama.cpp against all checkpoint tensor values and tokenizer IDs. Use six identical raw prompts, 128 output tokens, greedy sampling, BF16 weights and KV, concurrency 1 and 4, warm-up and three measured repetitions, serial engine execution under one Strix lease. Preserve per-request results, resolved settings, hashes, build logs, memory and contention evidence. Correctness precedes acceptance of speed ratios. Record every engine, including unsupported or failing arms; never substitute weights or eager vLLM settings.
Acceptance: all four demonstrably build and run the same model on Strix; exact prompt IDs and output counts; applicable correctness gate; repeated matched throughput/latency/memory with raw evidence; fresh implementation, mutation review, and operator gates. No predetermined speed winner. Links: #3043 and #3048 establish the prior 27B vLLM runtime, not this new model or benchmark.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read .agents/specs/strix-four-engine-qwen3-4b.md once it is committed, then review #3043 and #3048 for the prior Strix campaign context. Run the four-engine campaign on Strix using the six identical prompts and specified repetitions, preserving raw settings, outputs, hashes, logs, memory, and contention evidence. Done means every engine is recorded and the correctness and matched-performance acceptance gates are satisfied.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning, performance, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100