mudler / mudler/vllm.cpp

bench: extend matched Strix qualification to c1 c4 and c32

Open
#3,107 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: BACKEND-GATE-ROCM-SGLANG

Parent: #3053. Performance campaign: #3076. Owner: current Strix campaign operator.

The developer approved c1, c4, and c32 coverage on 9 September 2026, with a speed target at every tested concurrency. They also approved evenly repeating the original six raw prompts so c32 has enough requests. Keep Qwen3-4B BF16, 128-token greedy decoding, production pinned vLLM, patched SGLang, llama.cpp, and vllm.cpp. No model, precision, oracle-pin, or correctness waiver is included.

The existing harness binds six requests, four request slots, and 8192 KV tokens. Inspect each adapter and its public engine capacity APIs before implementation. Specify a versioned balanced request corpus used identically at c1/c4/c32, required capacity, occupancy evidence, refill/tail behavior, exact output ordering and token counts. Preserve the original six-request evidence as a different workload; do not compare old and expanded throughput as an optimization ratio.

Commit a scoped spec under .agents/specs before fresh test-first implementation. Add CPU regressions and independent mutations for concurrency/capacity, repeated request identities, prompt IDs, settings, missing or reordered outputs, and refusal paths. Require independent review and operator verification, then leased hardware correctness before accepted throughput. Report throughput, latency, and memory separately at each concurrency; retain failures rather than inventing c32 support.

Existing teardown, oracle repeatability, SDK integration, and comparator correctness blockers remain open. This scope does not authorize ignoring zombies, serialized callbacks, eager performance denominators, or claims based on requested rather than actual occupancy.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the scoped spec under .agents/specs, then inspect each adapter and its public engine capacity APIs. Define the versioned corpus, occupancy evidence, output checks, and CPU regressions described in the issue before leased-hardware verification. Done means c1/c4/c32 results are independently reviewed, failures are retained, and throughput, latency, and memory are reported separately.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, performance, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.