mudler/vllm.cpp

View on GitHub

a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features (GGUF, RadixAttention, Cache-aware scheduling, ...)

Stars: 423
Forks: 53
Open beginner issues: 0
Indexed issues: 351
Avg merge: 20h 26m
Merged PRs (30d): 310

Dominant language: C++
License: Apache-2.0
Last GitHub push: Sep 20, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed

Beginner friendly Everything

Open All

351 open issues indexed

fix(ENG-RUNNER-MODELSHAPE): block-table width aligns by 128/block_size, not 128/gcd(128, block_size) Open

Difficulty 4/5 3-5 days Newbie friendliness 68/100

mudler/vllm.cpp#2656 · Opened Sep 3, 2026
sync(UPSTREAM-SYNC-HEADPIN): six deferred PORT-NOW entries from tranche 81-120, each with the gate that makes it real Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

mudler/vllm.cpp#2655 · 1 comment · Opened Sep 3, 2026
record(MODEL-TEXT-qwen3-5-qwen3-5-for-causal-lm): the backbone-prefix comment cites a mapper that does not exist at ad5d29db70 Open

Difficulty 1/5 Under an hour Newbie friendliness 91/100

mudler/vllm.cpp#2654 · Opened Sep 3, 2026
fix(TOOLS-PARSER-BREADTH): lfm2 drops tool calls containing a nested quote or a bracket in a string Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#2653 · Opened Sep 3, 2026
fix(TOOLS-STREAMING-PARSER): an Inkling response with no thinking block never emits REASONING_END Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

mudler/vllm.cpp#2652 · Opened Sep 3, 2026
fix(SERVE-METRICS): a zero-output step records no scheduler stats, so the gauges freeze Open

Difficulty 4/5 3-5 days Newbie friendliness 56/100

mudler/vllm.cpp#2651 · Opened Sep 3, 2026
port(KERNEL-ATTN-MLA-SPARSE): MLA chunked context is the pre-b38e111d3e equal-share grid, not per-request packing Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

mudler/vllm.cpp#2650 · Opened Sep 3, 2026
port(SPEC-DRAFT-MODEL): async_scheduling_compatible refuses draft_model, which the pin advance makes wrong Open

Difficulty 3/5 1-2 days Newbie friendliness 58/100

mudler/vllm.cpp#2649 · Opened Sep 3, 2026
port(ENG-PRIORITY-SCHED): the X-Vllm-Priority request header is never read Open

Difficulty 3/5 1-2 days Newbie friendliness 76/100

mudler/vllm.cpp#2648 · Opened Sep 3, 2026
dots3-note: resized_size's detail / image_details / target_size overrides are not wired to the processor Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

mudler/vllm.cpp#2645 · Opened Sep 3, 2026
port(BACKEND-CUDA-COMP-FA): the MLA prefill priority list is the pre-bf2b45b5d6 order and takes no MLA dimensions Open

Difficulty 3/5 1-2 days Newbie friendliness 72/100

mudler/vllm.cpp#2640 · Opened Sep 3, 2026
port(KV-OFFLOAD): the offload identity is never parallelism-agnostic, so a cache cannot be reread under a different TP degree Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

mudler/vllm.cpp#2639 · Opened Sep 3, 2026
port(TOOLS-STREAMING-PARSER): Mistral is not on the parser-engine seam and keeps a second hand-written implementation Open

Difficulty 5/5 Over a week Newbie friendliness 42/100

mudler/vllm.cpp#2638 · Opened Sep 3, 2026
fix(TOOLS-STREAMING-PARSER): bare MESSAGE_HEADER text is dropped instead of emitted Open

Difficulty 3/5 1-2 days Newbie friendliness 76/100

mudler/vllm.cpp#2637 · Opened Sep 3, 2026
fix(ENG-RUNNER-MODELSHAPE): BlockTable::move_row leaves the vacated source row populated Open

Difficulty 2/5 1-3 hours Newbie friendliness 88/100

mudler/vllm.cpp#2636 · Opened Sep 3, 2026
A18: reference-audio delivery is a product decision, not a port — its only upstream caller is DubIt, which this tree does not ship Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#2634 · Opened Sep 3, 2026
A comment-only change to a checker cannot satisfy check-pr-size's mutation-evidence contract Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

mudler/vllm.cpp#2631 · Opened Sep 3, 2026
qwen4_exp QSA cooperative_topk cannot launch on NVIDIA Thor (sm_110): cluster misconfiguration Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#2626 · Opened Sep 3, 2026
Can the pinned vLLM serve our Qwen3.8-27B Q4_K_M GGUF through vllm-gguf-plugin, and displace llama.cpp as this arm's oracle? Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

mudler/vllm.cpp#2624 · 1 comment · Opened Sep 2, 2026
dots3-note vision tower: use_bias=true still refuses, because the shared MlpGateUpMethodBase seam has no bias arm Open

Difficulty 5/5 Over a week Newbie friendliness 42/100

mudler/vllm.cpp#2616 · 2 comments · Opened Sep 2, 2026
dots3-note vision MoE: the SOFTMAX router and the topk<2 arm are refused, because the self-normalizing combine needs a per-token divisor Open

Difficulty 5/5 Over a week Newbie friendliness 32/100

mudler/vllm.cpp#2615 · Opened Sep 2, 2026
OpRegistered is a presence probe, not a capability probe: kMlaPrefillAttention is registered on CUDA even when the kernel throws for lack of FA2 Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#2614 · Opened Sep 2, 2026
GDN prefill runs the sequential recurrence on CPU/ROCm/Vulkan where vLLM runs the chunked algorithm on every device — 2.3e-4 out, 2.2e-3 state, and it is 100% algorithmic Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#2612 · 1 comment · Opened Sep 2, 2026
HEADPIN follow-up: the run half at e126687a9a, its 290-entry PORT-NOW queue, and whether the head installs Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

mudler/vllm.cpp#2611 · Opened Sep 2, 2026
Qwen3.8-27B Q4_K_M: p4/14 is the CPU tier diverging across ARCHITECTURES, not ROCm; #2590 narrows to two steps Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

mudler/vllm.cpp#2608 · Opened Sep 2, 2026
check-agent-record's issue-ownership gate exits 0 and prints nothing when its snapshot is absent, which is always the case in CI Open

Difficulty 4/5 3-5 days Newbie friendliness 56/100

mudler/vllm.cpp#2606 · Opened Sep 2, 2026
Nothing refuses a comma in a doctest TEST_CASE name, and 1202 of them are already unselectable by -tc Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

mudler/vllm.cpp#2605 · Opened Sep 2, 2026
Half the tie-stability device sweep is a duplicate: VT_MOE_ROUTER_WARP does not change the dispatch above E = 256 Open

Difficulty 3/5 1-2 days Newbie friendliness 76/100

mudler/vllm.cpp#2604 · Opened Sep 2, 2026
The permissive HasCuda() skip lets a device-less CUDA build read green while measuring nothing Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#2603 · Opened Sep 2, 2026
EXL3 decoders take a const void* weight operand, so a wrongly typed argument no longer fails to compile Open

Difficulty 4/5 3-5 days Newbie friendliness 64/100

mudler/vllm.cpp#2597 · Opened Sep 2, 2026
ROCm gfx1151: the Q4_K_M arm diverges from the CPU arm at three near-tie steps on an identical prefix Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#2590 · Opened Sep 2, 2026
The EXL3 device arm labels never print: MESSAGE stringifies a const char* as a bool Open

Difficulty 2/5 1-3 hours Newbie friendliness 88/100

mudler/vllm.cpp#2587 · Opened Sep 2, 2026
A8: the spatiotemporal latent upsampler is a product decision, not a port — its output shape has no consumer above 8 frames Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#2584 · Opened Sep 2, 2026
Twenty more const uint16_t* casts over borrowed safetensors bytes remain after the six that landed Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#2579 · Opened Sep 2, 2026
Hybrid placement: routed-expert towers placed on the host load UNREPACKED, because quant_repack is per-load and reads the engine device Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#2568 · Opened Sep 2, 2026
--fit sizes a placement from ON-DISK bytes, so it can leave a tower on a device whose keep-quant set cannot hold it Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#2565 · Opened Sep 2, 2026
llama.cpp b10451 built for HIP emits degenerate output at n_gpu_layers=0 on gfx1151's host, while n_gpu_layers=99 is correct Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

mudler/vllm.cpp#2557 · Opened Sep 2, 2026
VT_KV_CACHE_F32=1 cannot complete a forward: reshape_and_cache refuses f32 cache against bf16 k/v Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#2548 · Opened Sep 2, 2026
qwen4_exp --device cuda is fluent but NOT token-exact against the CPU control, and the divergence is already there at prefill Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#2547 · 2 comments · Opened Sep 2, 2026
ROCm RmsNormPlusAdd (and DualRmsNormPlusRes) still weld every operand dtype to one T Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

mudler/vllm.cpp#2543 · Opened Sep 2, 2026
vt::RmsNorm: the kF16 ACTIVATION the seam admits and the CPU arm serves is refused by both device arms Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

mudler/vllm.cpp#2542 · Opened Sep 2, 2026
Qwen3.8-27B Q4_K_M is not token-exact vs llama.cpp b10451 on the CPU tier: 5 of 6 prompts diverge at rank-2 under 0.18 logits Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#2534 · 4 comments · Opened Sep 1, 2026
fix(SCHED-ASYNC-PREEMPT): a preemption leaves stale placeholders and the count goes negative under NDEBUG Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

mudler/vllm.cpp#2531 · 1 comment · Opened Sep 1, 2026
fix(SCHED-PRIORITY-PREEMPT): a preempted victim below the cursor silently drops the next request Open

Difficulty 3/5 1-2 days Newbie friendliness 72/100

mudler/vllm.cpp#2527 · 1 comment · Opened Sep 1, 2026
sync(PINPORT): steps 5-7 of the cdefd9d499 cycle -- port the queue, re-verify, advance the pin Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

mudler/vllm.cpp#2524 · 1 comment · Opened Sep 1, 2026
ROCm's device-weight budget is hipMemGetInfo's 64.00 GiB total, but every allocation on an integrated board is managed and tops out at a measured 58.000 GiB Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

mudler/vllm.cpp#2518 · Opened Sep 1, 2026
gfx1151 reports pageableMemoryAccess=0, so the host-slot expert lane can never serve on Strix Halo -- #2507's second condition is false, not satisfied Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

mudler/vllm.cpp#2515 · Opened Sep 1, 2026
LTX-2.5 adherence gap is not localized: nothing separates a divergent latent from a divergent VAE decode Open

Difficulty 4/5 3-5 days Newbie friendliness 42/100

mudler/vllm.cpp#2514 · Opened Sep 1, 2026
HIP builds fail to link when ccache is present in the container, and link without it Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mudler/vllm.cpp#2506 · Opened Sep 1, 2026
No device value names ROCm: the C ABI offers only auto/cpu/cuda, so an AMD box cannot pin its accelerator Open

Difficulty 5/5 Over a week Newbie friendliness 45/100

mudler/vllm.cpp#2505 · Opened Sep 1, 2026

mudler/vllm.cpp

Get new issues in your inbox