mudler/vllm.cpp

View on GitHub

a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features (GGUF, RadixAttention, Cache-aware scheduling, ...)

Stars: 423
Forks: 53
Open beginner issues: 0
Indexed issues: 351
Avg merge: 20h 26m
Merged PRs (30d): 310

Dominant language: C++
License: Apache-2.0
Last GitHub push: Sep 20, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed

Beginner friendly Everything

Open All

351 open issues indexed

ROCm: release captured graphs when instantiation fails Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#3049 · 2 comments · Opened Sep 7, 2026
MODEL-MM-GLM53-FLASH: establish a runnable pinned GLM5Next oracle Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#3047 · 1 comment · Opened Sep 7, 2026
Derive the KV budget from real startup memory profiling Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

mudler/vllm.cpp#3046 · Opened Sep 7, 2026
MODEL-MM-GLM53-FLASH: reconcile the merged vLLM implementation Open

Difficulty 3/5 1-2 days Newbie friendliness 68/100

mudler/vllm.cpp#3045 · 1 comment · Opened Sep 7, 2026
Establish the current pinned vLLM oracle on Strix Halo before packed-decode optimization Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

mudler/vllm.cpp#3043 · 7 comments · Opened Sep 7, 2026
27B dense keep-quant arm: decode residency exhausts device DRAM, e2e gate unmet Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#3042 · Opened Sep 7, 2026
Re-derive gfx1100 decode-graph activation on current main with device and replay proof Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

mudler/vllm.cpp#3041 · 2 comments · Opened Sep 7, 2026
gfx1151: establish valid rocprofv3 timestamps before kernel attribution Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#3040 · 1 comment · Opened Sep 7, 2026
perf(PERF-THOR-DECODE-PARITY): bring Qwen decode to vLLM parity on NVIDIA Thor Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

mudler/vllm.cpp#3038 · Opened Sep 7, 2026
ROCm KQuantGemmKWmmaQ4K/Q6K: redesign around a cooperative shared tile, matching llama.cpp's mul_mat_q mechanism Open

Difficulty 5/5 Over a week Newbie friendliness 38/100

mudler/vllm.cpp#3034 · Opened Sep 6, 2026
ROCm KQuantGemmKWmmaQ4K/Q6K prefill launches 16-48x more, smaller blocks than llama.cpp's mul_mat_q and is 4.9-10.6x slower Open

Difficulty 4/5 3-5 days Newbie friendliness 58/100

mudler/vllm.cpp#3032 · Opened Sep 6, 2026
SIGSEGV in InputBatch::add_request serving a text completion on a synthetic DeepSeek-V4 fixture, ~50% of runs Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

mudler/vllm.cpp#3027 · 1 comment · Opened Sep 6, 2026
A PNG or http(s) image is HTTP 500 on the Qwen3-VL and dots3-note chat seams, not 400 Open

Difficulty 4/5 3-5 days Newbie friendliness 66/100

mudler/vllm.cpp#3026 · Opened Sep 6, 2026
A2-5 is blocked: lifting the veto falsifies the device-mirror arm's own staleness justification, on the GB10 default Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

mudler/vllm.cpp#3025 · Opened Sep 6, 2026
SPEC-DFLASH2 A2-5: lift the async input-combine veto, the wave that captures the idle Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

mudler/vllm.cpp#3023 · Opened Sep 6, 2026
ROCm split-phase random sample: phase B launches 1024 threads onto 128-entry shared arrays Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

mudler/vllm.cpp#3022 · 1 comment · Opened Sep 6, 2026
No CI lane compiles a .hip: every src/vt/rocm change reaches main with zero build coverage, and #2957 landed that way Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

mudler/vllm.cpp#3021 · Opened Sep 6, 2026
gfx1151 falls back to the legacy QuantizeQ8KK: the cooperative Q8_K activation quantizer is gated on a strict gfx1100 prefix Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

mudler/vllm.cpp#3018 · Opened Sep 6, 2026
DotQ6K is scalar where DotQ4K uses hardware dot4, and materializes a 256-byte per-thread array; it serves ffn_down, 14.5% of Qwen3.8-27B's decode byte budget Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#3017 · Opened Sep 6, 2026
gfx1151: 61.7% of Qwen3.8-27B's per-token byte budget runs KQuantGemmK with 8-12 of 32 lanes idle, and #2086's cooperative arm is Q6_K-only Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

mudler/vllm.cpp#3016 · Opened Sep 6, 2026
gfx1151 decode: the 71.3% board-idle figure is a whole-leg average; windowed to the generate interval the board is 100% busy at 2872 MHz Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#3015 · 1 comment · Opened Sep 6, 2026
The hybrid-placement speed axis is untracked: its campaign issue #149 was deleted, and the fleet has no device that can measure it Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#3013 · Opened Sep 6, 2026
VT_SAMPLE_SPLIT is a new production env lever with no docs/ENVIRONMENT.md row Open

Difficulty 1/5 Under an hour Newbie friendliness 20/100

mudler/vllm.cpp#3012 · 1 comment · Opened Sep 6, 2026
check-role-discipline is red on main at 988482882: my merge commit did not name its task branch Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

mudler/vllm.cpp#3008 · 2 comments · Opened Sep 6, 2026
The block-wise FP8 GEMM owner #1189 is a deleted issue, and a shipped refusal string hands it to operators Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#3007 · Opened Sep 6, 2026
ROCm IQ4_XS keep-quant GEMM is 8.8x behind llama.cpp's WMMA path at the same CPU/GPU split Open

Difficulty 5/5 Over a week Newbie friendliness 44/100

mudler/vllm.cpp#3006 · Opened Sep 6, 2026
qwen4_exp: the cause of the three disagreeing token ids is still unexplained; selection flipping does not separate the forwards Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

mudler/vllm.cpp#2999 · Opened Sep 6, 2026
sync(UPSTREAM-SYNC-HEADPIN): step6-c1a credits the version read to a job whose own read returned 1 Open

Difficulty 2/5 1-3 hours Newbie friendliness 84/100

mudler/vllm.cpp#2997 · Opened Sep 6, 2026
clip mmproj: block_count from a user-supplied --mmproj reaches an unbounded resize on the production Qwen3-VL arm Open

Difficulty 3/5 1-2 days Newbie friendliness 76/100

mudler/vllm.cpp#2995 · Opened Sep 6, 2026
gpu_memory_utilization does not account for the DFlash2 draft speculative context, which is 1.25 GiB at max_num_seqs 8 Open

Difficulty 5/5 Over a week Newbie friendliness 45/100

mudler/vllm.cpp#2993 · Opened Sep 6, 2026
ResidentWeight's device-staging arm drops q8_0_aligned and repacked while guarding elem_kn_repacked Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#2992 · Opened Sep 6, 2026
test_opt_paged_engine: removing greedy_ids.npy deletes the whole SACRED gate into a green at 0 assertions Open

Difficulty 3/5 1-2 days Newbie friendliness 74/100

mudler/vllm.cpp#2988 · Opened Sep 5, 2026
Four SACRED gates carry #2805's mute switch: the self-determinism licence is inside an existence guard Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

mudler/vllm.cpp#2987 · Opened Sep 5, 2026
fix(KERNEL-GEMM-BF16): gate the toolkit install behaviourally -- the current test greps for a string an unreachable call satisfies Open

Difficulty 4/5 3-5 days Newbie friendliness 68/100

mudler/vllm.cpp#2986 · Opened Sep 5, 2026
check-agent-record stores a measurement of model-matrix.md, so re-anchoring a citation forces an edit to the checker and its test Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

mudler/vllm.cpp#2985 · Opened Sep 5, 2026
The Laguna 1.03x pairs a re-measured arm against a denominator taken 75 minutes earlier and never re-run Open

Difficulty 4/5 3-5 days Newbie friendliness 43/100

mudler/vllm.cpp#2983 · Opened Sep 5, 2026
test_cpu_x86_llamacpp_floor's contended-leg case is INTERMITTENT on an unchanged tree: one exit-4 NO_QUIET_WINDOW against the asserted exit-2 GIVING_UP, cause not established Open

Difficulty 3/5 1-2 days Newbie friendliness 67/100

mudler/vllm.cpp#2978 · 2 comments · Opened Sep 5, 2026
test_cpu_x86_llamacpp_floor reds as a function of machine load, on branches that never touched it Open

Difficulty 2/5 1-3 hours Newbie friendliness 76/100

mudler/vllm.cpp#2976 · Opened Sep 5, 2026
check-symbol-anchors deduplicates by (path, symbol), so its count is sites-minus-duplicates and a repair can UNMASK a stale citation Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

mudler/vllm.cpp#2971 · Opened Sep 5, 2026
GLM-5.3 non-flash on gfx1151: the forward dumps core, so no token has been observed since #2511 Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#2965 · 1 comment · Opened Sep 5, 2026
Decode is 96% GPU-busy and 96.4% of that is one GEMV, so the gap is kernel time not scheduling Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#2964 · 2 comments · Opened Sep 5, 2026
Four models bypass the typed HfConfig via cfg.raw, so normalization does not reach them Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

mudler/vllm.cpp#2963 · Opened Sep 5, 2026
Build the pinned vLLM oracle for ROCm/gfx1200 in Docker, with a benchmarking/profiling harness against vllm.cpp Open

Difficulty 5/5 Over a week Newbie friendliness 38/100

mudler/vllm.cpp#2961 · Opened Sep 5, 2026
Tenstorrent keep-quant dot kernels: every GGUF k-quant arm refuses on the P150, and Qwen3.8-27B Q4_K_M is the smallest artifact that fits Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#2959 · Opened Sep 5, 2026
docs/USAGE.md documents no checkpoint for Qwen3.8-27B bf16, the subject of a published benchmark whose gate exists only in prose Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

mudler/vllm.cpp#2951 · 1 comment · Opened Sep 5, 2026
Nothing holds main to the gates a branch must pass, and the red lands on the next unrelated branch Open

Difficulty 5/5 Over a week Newbie friendliness 42/100

mudler/vllm.cpp#2950 · 1 comment · Opened Sep 5, 2026
The served prompt-token count diverges from the client's corpus, and it biases our own throughput UP (refiles #1355) Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

mudler/vllm.cpp#2948 · Opened Sep 5, 2026
gfx1151: the vllm.cpp survey arm has never been measured on a post-#2511 head, so its fault rate is unknown Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

mudler/vllm.cpp#2944 · Opened Sep 5, 2026
#2777 deleted a recorded closed negative along with its stale preamble Open

Difficulty 1/5 Under an hour Newbie friendliness 85/100

mudler/vllm.cpp#2943 · 1 comment · Opened Sep 5, 2026
T36's op gate is not registered in tests/CMakeLists.txt, so it has never been built or run Open

Difficulty 1/5 Under an hour Newbie friendliness 88/100

mudler/vllm.cpp#2937 · Opened Sep 5, 2026

mudler/vllm.cpp

Get new issues in your inbox