vllm-project/production-stack

View on GitHub

vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

Stars: 2.6k
Forks: 503
Open beginner issues: 0
Indexed issues: 105
Avg merge: 4d 17h
Merged PRs (30d): 8

Dominant language: Python
License: Apache-2.0
Last GitHub push: Sep 18, 2026
Latest indexed: Sep 20, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: good first issue help wanted

Beginner friendly Everything

Open All

105 open issues indexed

bug: router keeps a deleted pod as a healthy backend when a DELETED event is missed during a watch reconnect Open

Difficulty 3/5 1-2 days Newbie friendliness 70/100

vllm-project/production-stack#1091 · Opened Sep 16, 2026
feature: Support Kubernetes Dynamic Resource Allocation (DRA) for GPU scheduling in Helm chart Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

vllm-project/production-stack#1087 · 1 comment · Opened Sep 13, 2026
bug: StaticServiceDiscovery validates one of four index-aligned lists, so a mismatch gives an IndexError at request time and a truncated health-check pass Open

Difficulty 2/5 1-3 hours Newbie friendliness 84/100

vllm-project/production-stack#1083 · 3 comments · Opened Sep 10, 2026
[Bug][Router] Unrecognized metric names yield an all-zero EngineStats that looks healthy — load-aware routing silently degrades to random Open

Difficulty 3/5 1-2 days Newbie friendliness 68/100

vllm-project/production-stack#1074 · Opened Sep 2, 2026
[Bug][Router] PrefixAwareRouter's below-threshold fallback uses stale QPS and starves one backend under session affinity Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

vllm-project/production-stack#1073 · 1 comment · Opened Sep 2, 2026
[Feature] Support hybrid-attention models (Qwen3.8-27B, Qwen3.5/3.6, GLM, Kimi-Linear) with LMCache MP mode (LMCacheMPConnector) Open

feature request

Difficulty 4/5 3-5 days Newbie friendliness 55/100

vllm-project/production-stack#1059 · 2 comments · Opened Aug 27, 2026
bug: LMCache cache server livenessProbe is incompatible with LMCache v0.5.4 image Open

bug

Difficulty 2/5 1-3 hours Newbie friendliness 76/100

vllm-project/production-stack#1057 · Opened Aug 27, 2026
Static-discovery /v1/models drops backend metadata — no way to advertise max_model_len Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

vllm-project/production-stack#1049 · 2 comments · Opened Aug 24, 2026
feature: servingEngineSpec to support statefulSets for modelSpec Open

feature request

Difficulty 5/5 Over a week Newbie friendliness 45/100

vllm-project/production-stack#1034 · Opened Aug 7, 2026
feature: Add Apache APISIX as an optional Gateway API controller Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

vllm-project/production-stack#1028 · Opened Aug 1, 2026
refactor: standardize route_request signature across all routers Open

Difficulty 4/5 3-5 days Newbie friendliness 68/100

vllm-project/production-stack#1022 · Opened Jul 28, 2026
KvawareRouter blocks the event loop per request (sync tokenizer fetch + /tokenize POST + controller lookup) — health probes starve, router CrashLoops under load Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

vllm-project/production-stack#1016 · 3 comments · Opened Jul 26, 2026
Router returns fatal 404 (not retryable 503) for a scaled-to-zero model when the router pod started while the pool was at 0 replicas Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

vllm-project/production-stack#1003 · Opened Jul 21, 2026
bug: The VLLMRuntime controller fails on every reconciliation when running on a cluster that does not have KEDA installed Open

bug

Difficulty 2/5 1-3 hours Newbie friendliness 72/100

vllm-project/production-stack#981 · 1 comment · Opened Jun 22, 2026
feature: add Uvicorn logging parameters to `vllm_router` Open

feature request

Difficulty 3/5 1-2 days Newbie friendliness 55/100

vllm-project/production-stack#967 · Opened Jun 3, 2026
feature: add first-class multi-node multiprocessing deployment support Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

vllm-project/production-stack#930 · 1 comment · 1 reaction · Opened Apr 26, 2026
feature: Implement Anthropic external provider (Phase 2) Open

feature request

Difficulty 4/5 3-5 days Newbie friendliness 58/100

vllm-project/production-stack#928 · 1 comment · 1 reaction · Opened Apr 22, 2026
bug: LMCache cacheserver OOMKilled Open

bug

Difficulty 3/5 1-2 days Newbie friendliness 58/100

vllm-project/production-stack#923 · 4 comments · 1 reaction · Opened Apr 17, 2026
feature: per-alias reasoning_effort for backward-compatible model migration Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

vllm-project/production-stack#911 · 1 reaction · Opened Apr 10, 2026
feature: crd ressources : split request and limit ressources Open

feature request

Difficulty 3/5 1-2 days Newbie friendliness 45/100

vllm-project/production-stack#901 · 2 comments · 1 reaction · Opened Mar 29, 2026
bug: lmCacheConfig with Blackwell 6000 pro and CUDA 13 Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 43/100

vllm-project/production-stack#893 · 4 comments · Opened Mar 24, 2026
Router default memory limit (500Mi) causes OOMKilled in real cluster deployments Open

Difficulty 3/5 1-2 days Newbie friendliness 58/100

vllm-project/production-stack#858 · 2 comments · Opened Feb 27, 2026
[Roadmap] vLLM Production Stack 2026 Roadmap Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

vllm-project/production-stack#855 · 12 comments · 9 reactions · Opened Feb 27, 2026
feature: Helm chart improvements Open

feature request

Difficulty 4/5 3-5 days Newbie friendliness 35/100

vllm-project/production-stack#840 · 2 comments · Opened Feb 19, 2026
Air gapped support Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

vllm-project/production-stack#838 · 3 comments · Opened Feb 17, 2026
feature: Add extra Vllm dashboard production oriented metrics Open

feature request

Difficulty 4/5 3-5 days Newbie friendliness 45/100

vllm-project/production-stack#828 · 2 comments · 2 reactions · Opened Feb 6, 2026
Lmcache Dashboard problem Open

bug

Difficulty 3/5 1-2 days Newbie friendliness 42/100

vllm-project/production-stack#818 · 1 comment · Opened Feb 2, 2026
Difference between lmcache/vllm-openai v0.3.12 vs v0.3.12-lightweight Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

vllm-project/production-stack#796 · 1 comment · Opened Jan 23, 2026
feature: global registry in helm chart Open

feature request

Difficulty 2/5 1-3 hours Newbie friendliness 48/100

vllm-project/production-stack#785 · Opened Jan 19, 2026
feature: allow multiple api keys in `servingEngineSpec.vllmApiKey` Open

feature request

Difficulty 3/5 1-2 days Newbie friendliness 48/100

vllm-project/production-stack#759 · 1 comment · 4 reactions · Opened Nov 19, 2025
bug: vllm-router during Disaggregated Prefill fails w/ `error: unrecognized arguments: --k8s-service-discovery-type pod-ip` Open

bug

Difficulty 2/5 1-3 hours Newbie friendliness 48/100

vllm-project/production-stack#746 · 1 comment · Opened Oct 31, 2025
Readiness probe for multi-node setup cannot be customized and might make the Ray head pod unready Open

bug

Difficulty 3/5 1-2 days Newbie friendliness 35/100

vllm-project/production-stack#742 · Opened Oct 29, 2025
Any reason for "latest" tag in lmcache/vllm-openai image? Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

vllm-project/production-stack#734 · 1 reaction · Opened Oct 14, 2025
bug: lmcache remote server consistently disconnected from client side Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

vllm-project/production-stack#723 · Opened Oct 6, 2025
feature: [RFC] llama.cpp CPU Backend Integration for Production Stack Open

feature request

Difficulty 5/5 Over a week Newbie friendliness 30/100

vllm-project/production-stack#713 · 1 comment · Opened Sep 24, 2025
bug: potential rotuer memory leak Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

vllm-project/production-stack#707 · Opened Sep 19, 2025
bug: [lmcache/lmstack-router] Router does not forward request after uncertain amount of time Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 45/100

vllm-project/production-stack#706 · 4 comments · Opened Sep 19, 2025
Feature Request: Add Prometheus Metrics for Monitoring Models via vLLM Router Open

feature request

Difficulty 5/5 Over a week Newbie friendliness 35/100

vllm-project/production-stack#699 · 7 comments · 20 reactions · Opened Sep 17, 2025
bug: router crashes because of OOM and prefix cache HashTrie memory usage is unbounded Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 38/100

vllm-project/production-stack#687 · 3 comments · 1 reaction · Opened Sep 8, 2025
vllm_router : add debug headers in the response Open

feature request

Difficulty 3/5 1-2 days Newbie friendliness 48/100

vllm-project/production-stack#683 · 1 comment · Opened Sep 5, 2025
feature: Support command and args in the servingEngineSpec.modelSpec[] Open

feature request

Difficulty 3/5 1-2 days Newbie friendliness 68/100

vllm-project/production-stack#682 · 1 comment · Opened Sep 5, 2025
bug: Can't pass body of request in swagger docs Open

feature request good first issue

Difficulty 3/5 1-2 days Newbie friendliness 35/100

vllm-project/production-stack#667 · 3 comments · Opened Aug 27, 2025
bug: TypeError: k8 service discovery fail to load config Open

bug

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

vllm-project/production-stack#660 · 1 comment · Opened Aug 25, 2025
Dynamic config reload gets stuck after invalid JSON; does not recover after fixing Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 48/100

vllm-project/production-stack#659 · 4 comments · 1 reaction · Opened Aug 25, 2025
feature: Check sleep mode only interacting with the /is_sleeping endpoint Open

feature request

Difficulty 3/5 1-2 days Newbie friendliness 45/100

vllm-project/production-stack#655 · 10 comments · Opened Aug 22, 2025
bug: [Benchmark] ValueError: Length mismatch in multi-round-qa benchmark DataFrame creation Open

bug

Difficulty 3/5 1-2 days Newbie friendliness 45/100

vllm-project/production-stack#646 · 5 comments · Opened Aug 15, 2025
[Roadmap] vLLM Production Stack roadmap for 2025 Q3 Open

Difficulty 5/5 Over a week Newbie friendliness 15/100

vllm-project/production-stack#640 · 1 comment · 9 reactions · Opened Aug 11, 2025
bug: Cancellations don't propagate through the vLLM router to the engines Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 38/100

vllm-project/production-stack#634 · 4 comments · Opened Aug 6, 2025
feature: Health check fails on vLLM instances with authentication – add API key handling Open

feature request

Difficulty 4/5 3-5 days Newbie friendliness 52/100

vllm-project/production-stack#631 · 3 comments · 2 reactions · Opened Aug 5, 2025
[RFC] Improvements on PD router Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

vllm-project/production-stack#630 · Opened Aug 4, 2025

vllm-project/production-stack

Get new issues in your inbox