mudler/vllm.cpp

View on GitHub

a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features (GGUF, RadixAttention, Cache-aware scheduling, ...)

Stars: 423
Forks: 53
Open beginner issues: 29
Indexed issues: 351
Avg merge: 19h 27m
Merged PRs (30d): 327

Dominant language: C++
License: Apache-2.0
Last GitHub push: Sep 20, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed

Beginner friendly Everything

Open All

350 open issues indexed

test_cuda_deepseek_v4's COMPUTE-on-CUDA case cannot discriminate the fused EXL3 MoE arm: doubling its output moves max |diff| by 4.6e-5 against a 1e-2 tolerance Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

mudler/vllm.cpp#2500 · Opened Sep 1, 2026
VLLM_CPP_HIP=ON configures without hipBLASLt and then fails the build 539 TUs later Open

Difficulty 4/5 3-5 days Newbie friendliness 66/100

mudler/vllm.cpp#2499 · Opened Sep 1, 2026
ROCm gfx1151: no quant-matched decode number against llama.cpp on Strix Halo Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#2497 · 4 comments · Opened Sep 1, 2026
main CI never completes: cancel-in-progress is true for push, so every run is cancelled by the next push (0 successes in 100 runs) Open

Difficulty 1/5 Under an hour Newbie friendliness 84/100

mudler/vllm.cpp#2491 · Opened Sep 1, 2026
qwen4_exp QSA widens q and the output gate to f32 where vLLM keeps the bf16 model dtype Open

Difficulty 4/5 3-5 days Newbie friendliness 68/100

mudler/vllm.cpp#2488 · 2 comments · Opened Sep 1, 2026
The LTX-2.5 text tower computes a [T, 262144] lm_head, a SoftCap and a full download that nothing reads, twice per render Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#2479 · Opened Sep 1, 2026
ENG-MULTIKV-BYNAME: MultiKvCacheIndex::num_groups()'s comment says it counts attn_kv's groups, and since 9e7621efc it counts every published cache's Open

Difficulty 1/5 Under an hour Newbie friendliness 92/100

mudler/vllm.cpp#2459 · Opened Sep 1, 2026
MODEL-DSV4: DeepseekV4PagedSwaPages indexes attn_kv with MultiKvCacheIndex::Find's FLAT index, which is a paged slot only because that model publishes no recurrent group Open

Difficulty 3/5 1-2 days Newbie friendliness 65/100

mudler/vllm.cpp#2456 · Opened Sep 1, 2026
DeepSeek-V4 cannot load on the default configuration: its own factory publishes fp8_ds_mla and ApplyCacheDType refuses it (no store/read, owed to W5) Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#2455 · 3 comments · Opened Sep 1, 2026
No production path gives an EXL3 checkpoint a paged KV cache: DeepseekV4ForwardExl3Paged has only test callers Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

mudler/vllm.cpp#2447 · 1 comment · Opened Aug 31, 2026
ROCm: EXL3 runs through two CPU reference operations Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#2433 · Opened Aug 31, 2026
origin/main fails its own gate ratchet at its own tip: no session can reach a green preflight Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

mudler/vllm.cpp#2428 · Opened Aug 31, 2026
toy_model_plugin's kToyInfo is positional over a ModelInfo whose leading fields are all bool, so a mid-struct insert misassigns SILENTLY Open

Difficulty 1/5 Under an hour Newbie friendliness 92/100

mudler/vllm.cpp#2420 · Opened Aug 31, 2026
MODEL-MM-GLM53-FLASH-CUDA: GLM-5.3-Flash's k-pool indexer has no device op, so the device arm is not a routing-only port Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#2415 · Opened Aug 31, 2026
Add DeepSeek V4 Flash Vision support Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

mudler/vllm.cpp#2411 · 1 comment · Opened Aug 31, 2026
ENG-MM-INPUT-PIPELINE: the five paths P2 made reachable without making complete Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#2408 · Opened Aug 31, 2026
Op provider: route runtime selection by full device Open

Difficulty 5/5 Over a week Newbie friendliness 28/100

mudler/vllm.cpp#2402 · Opened Aug 31, 2026
check-commit-style reports an unevaluable range as a style failure, and races a moving main Open

Difficulty 3/5 1-2 days Newbie friendliness 74/100

mudler/vllm.cpp#2400 · Opened Aug 31, 2026
The block-decoding gather arms for METAL, VULKAN, ROCM and TENSTORRENT Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#2394 · Opened Aug 31, 2026
Roadmap stores stale issue state after index retirement Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#2390 · Opened Aug 31, 2026
Jump-forward decoding ships a server flag, an ABI field and a FEATURES tick, and its capability function has no caller Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#2387 · 1 comment · Opened Aug 31, 2026
Benchmark: record AMD GPU clocks in leased ROCm runs Open

Difficulty 4/5 3-5 days Newbie friendliness 58/100

mudler/vllm.cpp#2381 · Opened Aug 31, 2026
ROCm: DFlash2 reaches host fallbacks and hangs gfx1151 Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

mudler/vllm.cpp#2377 · Opened Aug 31, 2026
vt CPU attention: the per-element dtype dispatch is 64.6% of the kernel, and 219 call sites share the defect Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

mudler/vllm.cpp#2376 · Opened Aug 31, 2026
documentation-checkpoint reds on main: four recent commits reached main without arriving on a task branch Open

Difficulty 4/5 3-5 days Newbie friendliness 42/100

mudler/vllm.cpp#2375 · Opened Aug 31, 2026
check-agent-record resolves record links against the worktree, so an untracked target gates green locally and red in CI Open

Difficulty 3/5 1-2 days Newbie friendliness 74/100

mudler/vllm.cpp#2369 · Opened Aug 30, 2026
agent-ready cannot pass anywhere: the checker-discovery loop always skips five arg-gated gates against the zero-skip READY contract Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#2367 · Opened Aug 30, 2026
VT_QWEN35_STAGE_RESERVE_BYTES undocumented — the third env var to red main in two days, and the checker cannot see duplicates Open

bug

Difficulty 1/5 Under an hour Newbie friendliness 88/100

mudler/vllm.cpp#2357 · Opened Aug 30, 2026
check-env-doc is RED on main again: #2349 landed VT_QWEN35_STAGE_RESERVE_BYTES undocumented Open

Difficulty 1/5 Under an hour Newbie friendliness 85/100

mudler/vllm.cpp#2356 · Opened Aug 30, 2026
conditioning.connector is 23.6% of an LTX-2.5 render and nothing says whether that is the weights or the arithmetic Open

Difficulty 4/5 3-5 days Newbie friendliness 68/100

mudler/vllm.cpp#2354 · Opened Aug 30, 2026
ModelRegistry::Forward's multi-cache refusal names ONE owner where THREE architectures now arrive, and never names the arriving architecture Open

Difficulty 2/5 1-3 hours Newbie friendliness 84/100

mudler/vllm.cpp#2353 · Opened Aug 30, 2026
check-env-doc is red on main a THIRD time today: VT_QWEN35_STAGE_RESERVE_BYTES from #2349, and the recurrence is the finding Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

mudler/vllm.cpp#2350 · Opened Aug 30, 2026
Request: musl-vulkan binary Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#2299 · Opened Aug 29, 2026
tp>1 prefill is host-bound: per-token fp4 GEMM + per-layer H2D activation staging Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#2144 · Opened Aug 28, 2026
TT eager decode spends the wall on host staging, not device kernels: cache resolved handles, bulk the EnsureDevice2D element loop, batch per-layer staging Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#2107 · Opened Aug 27, 2026
ROCm decode: #1863's 4b1154bc5 baseline (18.4 / 13.0 tok/s) does not reproduce on the same board Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

mudler/vllm.cpp#2095 · Opened Aug 27, 2026
BACKEND-DISTRIBUTED-TP-CLEAN: tp>1 fp8w tail-MLP shard host-dequantizes every layer (~1.7 s/layer) — make it a per-rank device-resident W8A16 GEMM Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#2058 · Opened Aug 27, 2026
index row for #1954 wrongly claims #907 documents the same test and assertion Open

Difficulty 2/5 1-3 hours Newbie friendliness 78/100

mudler/vllm.cpp#2013 · Opened Aug 26, 2026
QUANT-QWEN38-27B-NVFP4-ARM: qwen36 compressed-tensors mixed-precision FP8 group_0 is refused, blocking an apples-to-apples vllm.cpp vs vLLM comparison Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

mudler/vllm.cpp#1959 · Opened Aug 26, 2026
test_backend_cross_device: MoeSiluMul bf16 exactness reds on gfx1200 ROCm too, the ROCm counterpart of #1802 Open

Difficulty 1/5 Under an hour Newbie friendliness 86/100

mudler/vllm.cpp#1954 · 2 comments · Opened Aug 26, 2026
ROCm ports zero I-quant formats, and porting a complex-vec_dot one would test whether #1910's nwarps split transfers Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

mudler/vllm.cpp#1940 · Opened Aug 25, 2026
RocmPlatform::needs_weight_staging() is stale-false, so the #1123/#1870 device-fit refusal never runs on ROCm Open

Difficulty 5/5 Over a week Newbie friendliness 38/100

mudler/vllm.cpp#1934 · Opened Aug 25, 2026
ENG-WEIGHT-OFFLOAD on ROCm: four measured device facts from a throwaway gfx1200 spike, including a non-monotonic budget cliff and hipMallocManaged not migrating Open

Difficulty 4/5 3-5 days Newbie friendliness 42/100

mudler/vllm.cpp#1914 · Opened Aug 25, 2026
Weights are never made device-resident at load Open

Difficulty 5/5 Over a week Newbie friendliness 38/100

mudler/vllm.cpp#1897 · Opened Aug 25, 2026
QuantizeQ8KK is thread-per-superblock: 95.5us/call against llama.cpp's 1.6us, and 41% of the ROCm decode gap Open

Difficulty 4/5 3-5 days Newbie friendliness 42/100

mudler/vllm.cpp#1876 · 2 comments · Opened Aug 24, 2026
VT_GGUF_KEEP_QUANT=0 is unreachable on a 16 GiB ROCm card: the bf16 fallback OOMs with an allocator throw, not a refusal that names the cause Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#1870 · Opened Aug 24, 2026
ROCm decode is 2.4x behind llama.cpp on a DENSE model (Ornith-1.5-9B, qwen35): the gap is not MoE-specific Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#1863 · 2 comments · Opened Aug 24, 2026
Target-only Qwen3.8 W4 sampled decode selects mixed-dtype packed GDN Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#1842 · Opened Aug 24, 2026
Gemma-4 FP8 WMMA expert GEMM for M>1 prefill on gfx1201 Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

mudler/vllm.cpp#1762 · 1 comment · Opened Aug 23, 2026
Building error Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

mudler/vllm.cpp#1761 · Opened Aug 23, 2026

mudler/vllm.cpp

Get new issues in your inbox