InternLM/lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Stars: 8.1k
Forks: 748
Open beginner issues: 0
Indexed issues: 546
Avg merge: 6d 2h
Merged PRs (30d): 54

Dominant language: Python
License: Apache-2.0
Last GitHub push: Sep 16, 2026
Latest indexed: Sep 17, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed

Open All

546 open issues indexed

[Bug] Does lmdeploy support deploying glm-5 on A800 or A100, or are there any plans to support it? Open

InternLM/lmdeploy#4407 · 10 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[Feature] Add qwen3.5 fp8 support to Turbomind Open

InternLM/lmdeploy#4428 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[Bug] v0.12.2-cu12.8部署qwen3.5的时候请，求任何内容出现 illegal memory Open

InternLM/lmdeploy#4436 · 5 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Bug] MXFP4 Bug Open

InternLM/lmdeploy#4440 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Feature] Integrate Mooncake Transfer Engine with TurboMind for PD disaggregation and remote KV Open

InternLM/lmdeploy#4443 · 5 comments · 2 reactions · 0 assignees · Indexed Sep 17, 2026
[Feature] 现在可以支持Qwen3.5 4bit 量化吗？ Open

InternLM/lmdeploy#4464 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Bug] 我用了最新版的latest-cu128,运行GLM-4.7-Flash，在opencode中使用报了下面的错误！ Open

InternLM/lmdeploy#4475 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[Bug] 为什么qwen3VL的logits、last_hidden_state不支持返回呢？ Open

InternLM/lmdeploy#4479 · 3 comments · 1 reaction · 2 assignees · Indexed Sep 17, 2026
[Bug] lmdeploy run qwen3.5-122b-a10b-awq , which transformers version? Open

InternLM/lmdeploy#4484 · 3 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Bug] corrupted size vs. prev_size in fastbins Open

InternLM/lmdeploy#4489 · 3 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Bug] 使用lmdeploy在5090上cuda版本12.8，cuda toolkit同12.8 Open

InternLM/lmdeploy#4491 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Bug] 工具调用报错asyncio.exceptions.CancelledError: Cancelled by cancel scope 7fe9b6f1ed10 Open

InternLM/lmdeploy#4492 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Feature] Support MiniMax-M2.7 in TurboMind engine Open

backlog

InternLM/lmdeploy#4527 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[Feature] support DFlash: Block Diffusion for Flash Speculative Decoding Open

planned feature

InternLM/lmdeploy#4530 · 3 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Feature] TurboMind backend missing VLM support for DeepSeek-VL2, Llama4, Gemma3, Qwen3-VL Open

InternLM/lmdeploy#4553 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Feature] 0.12.3现在支持Qwen3.6-35B-A3B的量化版本了吗？比如4bit的量化版本？ Open

InternLM/lmdeploy#4556 · 12 comments · 1 reaction · 1 assignee · Indexed Sep 17, 2026
[Bug] Qwen3.6-27B-AWQ out of memory with TurboMind engine on V100 Open

InternLM/lmdeploy#4558 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[Feature] TurboMind后端支持视觉模型 Open

InternLM/lmdeploy#4562 · 3 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Feature] TurboMind后端支持glm5.1 awq在A100/A800上推理吗 Open

InternLM/lmdeploy#4612 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[Feature] TurboMind后端什么时候计划开始支持deepseek v4 flash？ Open

InternLM/lmdeploy#4658 · 0 comments · 1 reaction · 0 assignees · Indexed Sep 17, 2026
[Feature] v0.13.0版本AWQ 是否支持 Qwen3-VL? Open

InternLM/lmdeploy#4673 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[Bug] Check failed: data_type_v<T> == dtype_ (f16 vs. bf16) Open

InternLM/lmdeploy#4686 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Bug] /update_weights deserializes request-controlled pickle data before validation Open

InternLM/lmdeploy#4698 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[Bug] 0.14.0版本能否支持对Qwen3.5系列模型的量化？ Open

InternLM/lmdeploy#4720 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Feature] lmcache support? Open

InternLM/lmdeploy#4750 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Unauthenticated SSRF via image_url in the OpenAI-compatible chat endpoint (redirect bypass of the private-IP guard) Open

InternLM/lmdeploy#4761 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[Feature] qwen3.5可以turbo后端可以开启prefill cache吗 Open

InternLM/lmdeploy#4775 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Feature] DeepSeek-V4 MoE is Blackwell-only via DeepGEMM — is a non-Blackwell path wanted? Open

InternLM/lmdeploy#4824 · 9 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Feature] 现在最新的0.15.0可以支持最新的deepseek-v4-flash-0731吗？ Open

InternLM/lmdeploy#4832 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[Feature] sm75 can support awq/gptq/fp8/w8a8 in turbomind Open

InternLM/lmdeploy#4865 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Feature] 当前qwen-next等SSM类模型prefix cache缓存命中率低 Open

InternLM/lmdeploy#4879 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Bug] pytorch backend silently hangs under concurrent decoding with qwen3_5_mtp speculative decoding Open

InternLM/lmdeploy#4883 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Feature] 什么时候turbomind能够开始支持MTP或者DFlash之类的加速功能？ Open

InternLM/lmdeploy#4889 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Bug] PyTorch engine: Qwen3.5 hybrid-GDN + AWQ checkpoint crashes on TP>1 - 'AwqLinear' object has no attribute 'block_size' Open

InternLM/lmdeploy#4899 · 0 comments · 4 reactions · 0 assignees · Indexed Sep 17, 2026
[Feature] 咱们最新版本的lmdeploy可以加载这个 Qwen3.8-Flash吗？ Open

InternLM/lmdeploy#4905 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[Bug] Mooncake external KV keys omit KV format and weights lineage, with no default tenant isolation Open

InternLM/lmdeploy#4930 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Bug] input_embeddings are omitted from prefix-cache identity, causing wrong KV reuse Open

InternLM/lmdeploy#4933 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[Bug] TurboMind SIGSEGV crash-loop triggered by session_len truncation path (deterministic libc offset across 3 crashes) Open

InternLM/lmdeploy#4944 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Bug] Slower decode speed in v0.17 when compared to v0.14 Open

InternLM/lmdeploy#4953 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Feature] 现在lmdeploy能支持模型GLM-5.3-Flash-W4A16-MTP运行吗？ Open

InternLM/lmdeploy#4958 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[Enhance] Control token should not be treated as special token in user messages Open

InternLM/lmdeploy#4960 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Bug] Open

awaiting response

InternLM/lmdeploy#4962 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Bug] Untrusted `migration_request` can terminate the DistServe Decode EngineLoop and cause persistent denial of service Open

InternLM/lmdeploy#4965 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Bug] DistServe Proxy requests can leak Prefill scheduler metadata and OOM the Prefill engine Open

InternLM/lmdeploy#4967 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Bug] Mooncake async migration can hang forever and make Decode unavailable Open

InternLM/lmdeploy#4971 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Bug] Proxy connection warmup can accumulate unbounded PD connection wait tasks and OOM Open

InternLM/lmdeploy#4972 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026

A short digest of beginner-friendly GitHub issues.