ModelEngine-Group/unified-cache-management

View on GitHub

Persist and reuse KV Cache to speedup your LLM.

Stars: 334
Forks: 119
Open beginner issues: 0
Indexed issues: 52
Avg merge: 1d 15h
Merged PRs (30d): 82

Dominant language: C++
License: MIT
Last GitHub push: Sep 20, 2026
Latest indexed: Sep 19, 2026
Contributing guide: No contributing guide
Code of conduct: Code of conduct
Beginner labels: No beginner labels indexed

Beginner friendly Everything

Open All

52 open issues indexed

[Release]: Release checklist for v0.9.0 Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

ModelEngine-Group/unified-cache-management#1408 · Opened Sep 19, 2026
[Feature]: Lookup-triggered first-shard prefetch for CacheStore Open

feature request performance

Difficulty 5/5 Over a week Newbie friendliness 38/100

ModelEngine-Group/unified-cache-management#1382 · Opened Sep 16, 2026
[Feature]: NUMA-aware rank-partitioned shared CacheBuffer Open

feature request performance

Difficulty 5/5 Over a week Newbie friendliness 35/100

ModelEngine-Group/unified-cache-management#1381 · Opened Sep 16, 2026
[Feature] MiniMax M3 适配 Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

ModelEngine-Group/unified-cache-management#1378 · Opened Sep 15, 2026
[Feature]: Posix Store 多后端探测与高可用 Open

documentation

Difficulty 5/5 Over a week Newbie friendliness 20/100

ModelEngine-Group/unified-cache-management#1376 · Opened Sep 15, 2026
[Feature]: UCM Store integration with HiCache for Hybrid models Open

feature request

Difficulty 5/5 Over a week Newbie friendliness 25/100

ModelEngine-Group/unified-cache-management#1373 · Opened Sep 15, 2026
[Feature]: Support multimodal input kv cache Open

feature request

ModelEngine-Group/unified-cache-management#1370 · 1 assignee · Opened Sep 14, 2026
[Release]: Release checklist for v0.8.0rc1 Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

ModelEngine-Group/unified-cache-management#1363 · Opened Sep 11, 2026
UCM release: blocked upstream capabilities Open

Difficulty 5/5 Over a week Newbie friendliness 15/100

ModelEngine-Group/unified-cache-management#1360 · Opened Sep 11, 2026
[Feature]: Support Minimax-M3 with UCM on vLLM and vLLM-Ascend Open

feature request

Difficulty 5/5 Over a week Newbie friendliness 30/100

ModelEngine-Group/unified-cache-management#1358 · Opened Sep 11, 2026
[Feature]: Support Kimi-K3 with UCM on vLLM and vLLM-Ascend Open

feature request

Difficulty 5/5 Over a week Newbie friendliness 25/100

ModelEngine-Group/unified-cache-management#1357 · Opened Sep 11, 2026
[Feature]: Support DeepSeek-V4-Pro-0813 with UCM on vLLM and vLLM-Ascend Open

feature request

Difficulty 5/5 Over a week Newbie friendliness 35/100

ModelEngine-Group/unified-cache-management#1356 · Opened Sep 11, 2026
[Feature]: Support DeepSeek-V4-Flash-0731 with UCM on vLLM and vLLM-Ascend Open

feature request

Difficulty 5/5 Over a week Newbie friendliness 35/100

ModelEngine-Group/unified-cache-management#1355 · Opened Sep 11, 2026
[Feature]: Support GLM-5.3-Flash with UCM on vLLM and vLLM-Ascend Open

feature request

Difficulty 5/5 Over a week Newbie friendliness 35/100

ModelEngine-Group/unified-cache-management#1354 · 1 comment · Opened Sep 11, 2026
[Bug]: In PD decoupled deployment, using toy_proxy_server to send proxy requests will trigger an error when the `min_tokens` parameter is added, and the proxy service cannot process requests normally. Open

bug

Difficulty 2/5 1-3 hours Newbie friendliness 30/100

ModelEngine-Group/unified-cache-management#1348 · Opened Sep 9, 2026
UCM 2026 Q4 RoadMap Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

ModelEngine-Group/unified-cache-management#1314 · Opened Sep 1, 2026
[Feature]: Performance Optimization of the Multi-Rank Shared-Cache Load Path Open

feature request performance

ModelEngine-Group/unified-cache-management#1265 · 1 assignee · Opened Aug 21, 2026
Clarification on CacheStore buffer validation (-50000 too small buffer) for non-MLA models and the hard-coded 1024 floor Open

Difficulty 4/5 3-5 days Newbie friendliness 42/100

ModelEngine-Group/unified-cache-management#1261 · Opened Aug 20, 2026
[Doc]: add new docs website Open

documentation

Difficulty 5/5 Over a week Newbie friendliness 25/100

ModelEngine-Group/unified-cache-management#1258 · Opened Aug 20, 2026
[Draft][Feature]: UCM inference-sim tool (Toolkit sub-issue) Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

ModelEngine-Group/unified-cache-management#1225 · Opened Aug 12, 2026
[Feature][ToolKit]: 前缀重复率统计工具（字符级/token级/block级） Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

ModelEngine-Group/unified-cache-management#1221 · 1 reaction · Opened Aug 12, 2026
[Feature][ToolKit]: KV hash-directory recovery tool (recreate shards without restart) Open

Difficulty 5/5 Over a week Newbie friendliness 45/100

ModelEngine-Group/unified-cache-management#1220 · Opened Aug 12, 2026
[Feature]: UCM kv-calculator tool (Toolkit sub-issue) Open

feature request

Difficulty 5/5 Over a week Newbie friendliness 48/100

ModelEngine-Group/unified-cache-management#1217 · 2 comments · Opened Aug 10, 2026
[Feature]: UCM metrics-view tool (Toolkit sub-issue) Open

feature request

Difficulty 5/5 Over a week Newbie friendliness 35/100

ModelEngine-Group/unified-cache-management#1216 · Opened Aug 10, 2026
[Draft][Feature]: UCM ttft-analyze tool (Toolkit sub-issue) Open

feature request

Difficulty 4/5 3-5 days Newbie friendliness 48/100

ModelEngine-Group/unified-cache-management#1215 · Opened Aug 10, 2026
[Feature]: UCM bandwidth test tool (Toolkit sub-issue) Open

feature request

Difficulty 5/5 Over a week Newbie friendliness 42/100

ModelEngine-Group/unified-cache-management#1214 · 1 comment · Opened Aug 10, 2026
[Feature]: Global Storage Open

feature request

ModelEngine-Group/unified-cache-management#1213 · 1 comment · 1 assignee · Opened Aug 10, 2026
[Feature]: Transfer Engine Open

feature request

ModelEngine-Group/unified-cache-management#1212 · 1 comment · 1 assignee · Opened Aug 10, 2026
[Feature]: Refactor metrics-view in toolkit Open

feature request

Difficulty 5/5 Over a week Newbie friendliness 25/100

ModelEngine-Group/unified-cache-management#1211 · 1 comment · Opened Aug 10, 2026
[Feature]: Proxy Open

feature request

ModelEngine-Group/unified-cache-management#1210 · 1 comment · 1 assignee · Opened Aug 10, 2026
[Feature]: UCM artifact Open

feature request

Difficulty 5/5 Over a week Newbie friendliness 25/100

ModelEngine-Group/unified-cache-management#1209 · 1 comment · Opened Aug 10, 2026
[Feature]: ToolKit Open

feature request

Difficulty 5/5 Over a week Newbie friendliness 25/100

ModelEngine-Group/unified-cache-management#1208 · 1 comment · Opened Aug 10, 2026
[Feature]: Connector Open

feature request

Difficulty 5/5 Over a week Newbie friendliness 30/100

ModelEngine-Group/unified-cache-management#1207 · 1 comment · Opened Aug 10, 2026
[Feature]: CICD Open

feature request

ModelEngine-Group/unified-cache-management#1206 · 1 assignee · Opened Aug 10, 2026
[Feature] UCM V2 Architecture Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

ModelEngine-Group/unified-cache-management#1203 · Opened Aug 7, 2026
[Feature]: UcmPipelineStore support YuanRong|Posix Open

misc

Difficulty 5/5 Over a week Newbie friendliness 35/100

ModelEngine-Group/unified-cache-management#1177 · Opened Jul 30, 2026
[Bug]: In the vllm-ascend + ucm environment, ucm has no cache hit rate Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 45/100

ModelEngine-Group/unified-cache-management#1078 · 2 comments · Opened Jul 8, 2026
[Feature]: Introduce FFTS into p2p transport Open

feature request

Difficulty 5/5 Over a week Newbie friendliness 35/100

ModelEngine-Group/unified-cache-management#1061 · 2 comments · Opened Jun 29, 2026
[Feature]: # [Question] Does UCM support Qwen2.5-14B-Instruct-1M at extended context (>32K)? Open

feature request

Difficulty 4/5 3-5 days Newbie friendliness 42/100

ModelEngine-Group/unified-cache-management#948 · Opened May 6, 2026
[Bug]: 1P1D部署时，并发25出现decode节点报错：RPC call to sample_tokens timed out Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

ModelEngine-Group/unified-cache-management#939 · Opened Apr 28, 2026
[Bug]: UCM reports errors in multi-concurrency scenarios with long sequences (120K). Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 45/100

ModelEngine-Group/unified-cache-management#928 · 4 comments · Opened Apr 22, 2026
[Bug] Incorrect chunk hit bitmask indexing in UCMBlendConnector._get_req_chunk_hit when prefix cache partially hits Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

ModelEngine-Group/unified-cache-management#906 · Opened Apr 9, 2026
[Bug]: ucm0.5.0rc1同时5并发时报错 Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 32/100

ModelEngine-Group/unified-cache-management#900 · 11 comments · Opened Apr 7, 2026
[Bug]: ucm0.5.0rc1对于双节点的共享存储报错 Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

ModelEngine-Group/unified-cache-management#893 · 2 comments · Opened Apr 3, 2026
[Usage]: 请问0.5.0rc1支持双节点部署吗？ Open

usage

Difficulty 4/5 3-5 days Newbie friendliness 25/100

ModelEngine-Group/unified-cache-management#889 · 5 comments · Opened Apr 2, 2026
[Bug] Incorrect token generation after prefix cache hit (UCMBlendConnector, possible KV cache misalignment) Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 42/100

ModelEngine-Group/unified-cache-management#867 · 6 comments · Opened Mar 25, 2026
[Usage]: import ucm_custom_ops报错ModuleNotFoundError: No module named 'ucm_custom_ops' Open

usage

Difficulty 4/5 3-5 days Newbie friendliness 30/100

ModelEngine-Group/unified-cache-management#804 · 5 comments · Opened Mar 11, 2026
[Feature]: Open

feature request

Difficulty 5/5 Over a week Newbie friendliness 25/100

ModelEngine-Group/unified-cache-management#737 · 1 comment · Opened Feb 5, 2026
[Installation]: uv pip install --system --no-build-isolation uc-manager安装失败 Open

installation

Difficulty 4/5 3-5 days Newbie friendliness 40/100

ModelEngine-Group/unified-cache-management#736 · 4 comments · Opened Feb 5, 2026
[Usage]: Sparse 算法在npu支持哪些？ Open

usage

Difficulty 4/5 3-5 days Newbie friendliness 35/100

ModelEngine-Group/unified-cache-management#697 · 5 comments · Opened Jan 28, 2026

ModelEngine-Group/unified-cache-management

Get new issues in your inbox