[RoadMap][Call For Contributions] Mooncake Store V3 Roadmap

### Milestone 1: Core Architecture Refactor & Decoupling
This milestone focuses on foundational architectural changes to improve modularity, flexibility, and prepare for future scaling.

- [ ] (TE/Store Separation): Decouple the TE (Task/Tensor Engine) and Store components into separate, independent packages.
- [x] (Client/Worker Decoupling): Decouple the dummy client from the worker to remove strong dependencies. @YiXR
- https://github.com/kvcache-ai/Mooncake/pull/1084
- https://github.com/kvcache-ai/Mooncake/pull/1122
- #1146
- https://github.com/kvcache-ai/Mooncake/pull/1177
- https://github.com/kvcache-ai/Mooncake/pull/1206
- [ ] (Flexible Deployment): Update the Store to support various flexible deployment models, such as client-only, client + master, etc. 
- [ ] (Tensor-native APIs): Put/Get Tensor APIs contain TP rank and model info.
- #1127
- [ ] (Multiple Replicas): Support a replica list for each KV item.
- #1148
- [ ] (Layer-wise Storage): Maintaining layer info on the Mooncake side, supports stream transfer and communication overlap.

### Milestone 2: Master Service Enhancements
This milestone enhances the Master component to support new storage architectures and routing logic.

- [ ] (Key-based Routing): Implement new key-based routing capabilities in the Master service. 
- [ ] (Metadata Adaptation - Storage): Adapt the Master's metadata management to support the new multi-level storage architecture.
- [ ] (Recovery) kv metadata persistency https://github.com/kvcache-ai/Mooncake/pull/1451
- [ ] (KVCache Awareness Interface) Exposes hit ratio for different layers.
- [ ] (Metadata Adaptation - HA): Upgrade metadata schema and logic to meet new High Availability (HA) requirements. 
- Snapshot
- https://github.com/kvcache-ai/Mooncake/issues/1150
- https://github.com/kvcache-ai/Mooncake/pull/1381
- https://github.com/kvcache-ai/Mooncake/pull/1431
- https://github.com/kvcache-ai/Mooncake/pull/1465
- https://github.com/kvcache-ai/Mooncake/pull/1739
- Oplog
- https://github.com/kvcache-ai/Mooncake/issues/1200
- https://github.com/kvcache-ai/Mooncake/pull/1451
- https://github.com/kvcache-ai/Mooncake/pull/1515
- Refactoring HA
- https://github.com/kvcache-ai/Mooncake/issues/1648
- https://github.com/kvcache-ai/Mooncake/pull/1678
- https://github.com/kvcache-ai/Mooncake/pull/1722
- https://github.com/kvcache-ai/Mooncake/pull/1777
- [ ] (Multi-tenant): Support Multi-tenant with different models, users and auth keys

### Milestone 3: Worker: Multi-Level Storage Architecture
This is a major epic to build the next-generation multi-level storage system within the Worker.

- 3.1: Abstraction & Caching
- [ ] (Storage Abstraction Layer): Design and implement the core abstraction layer for multi-level storage. 
- [ ] (Cache Scheduling Interface): Design the abstract interface for cache scheduling logic. 
- [x] (Eviction Logic): Implement basic data eviction logic within the new storage architecture.
- @Vincent-Bo-ali #1028
- [ ] (LRU Cache): Implement an LRU (Least Recently Used) policy as the default cache scheduling strategy. 
- [ ] (Local Client Cache): Keep a local cache for better performance.
- #1062

- 3.2: Storage Backend Implementation
- [ ] (DRAM Adaptation): Adapt the storage layer for DRAM, including support for NUMA affinity. 
- [ ] (SSD Adaptation): Adapt the storage layer for SSDs, enabling local external storage read/write capabilities. 
- #968
- #1031
- #1054
- #1203
- #1284
- #1319
- #1456
- [ ] (VRAM Adaptation): Adapt the storage layer to utilize VRAM. 
- [ ] (Huawei NPU Adaptation): Implement support for Huawei NPUs (H2D). 
- @ascend-direct-dev

- 3.3: Elastic KVCache Storage
- [ ] (KVCache Migration): Move KVCache between Mooncake client.
- #1326
- [ ] (Data Replica): Dynamic Replication
- #1100

### Milestone 4: Worker: Networking & Elasticity
This milestone focuses on refactoring worker communication and enabling resource elasticity.

- [ ] (RPC Refactor): [Phase 1] Refactor the worker's read/write logic to replace RDMA with RPC-based communication. 
- [x] (Barex Transport Support): Support Alibaba barex transport in TE for Mooncake Store.
- #1045
- [ ] (Resource Elasticity): Implement single-worker resource elasticity. 
- [ ] (Event‑driven completion): Provide an option to using event-driven notification worker instead of busy-polling.
- #1033
- #1053
- [ ] (IPv6 Support): Support IPv6 in client, master and metadata server.
- #1043
- #1067

### Milestone 5: Deployment & Operations
This milestone covers K8s integration (i.e., RBG, https://github.com/sgl-project/rbg) and build process improvements.

- [ ] (K8s Autoscaling): Implement support for Kubernetes-based autoscaling of worker and dummy client instances.
- [ ] (Scenario-based Builds): Implement a build system capable of producing different worker binaries optimized for different scenarios. 
- [ ] (Integration With AI Configurator): Use AI Configurator for better measuring Resource workers and other configurations.
- [ ] (Deployment Documentation & Guides): Create comprehensive, up-to-date deployment documentation and step-by-step setup guides to simplify installation and configuration for all environments.

### Milestone 6: Conductor
This feature implements a standalone or co-located conductor for use by the Gateway. @zhongzhouTan-coder @yejj710 @Liziqi-77 @Keithwwa @Asher-XunZhang
- [x] (Basic design) : Discuss and design the standard specifications for KV indexer and kv-events
- https://github.com/kvcache-ai/Mooncake/issues/1403
- https://github.com/kvcache-ai/Mooncake/issues/1527
- [ ] (KV indexer): Implement a Mooncake KV-Store Indexer service (include /quey, /query_by_hash ..etc)
- [ ] (KV publisher): support kv-event publisher in Mooncake store
- [ ] (Service registration): support two methods for service registration: either through static configuration files or dynamic service registration APIs
- [ ] (KV events): Support consuming KV events generated by inference frameworks and Mooncake

### Milestone 7: Hardware Compatibility
- [ ] (AMD) HIP, ROCm, MORI
- #1154
- #1179
- #1189
- [ ] (Moore Thread)
- #973
- #1176
- [ ] (AWS EFA)
- #1509
- #1523
- #1526

### Milestone 8: Pytorch Eco-system
- [ ] (Support Tensor Attributions)
- [ ] (Native torch format offload)

### Milestone 9: CI & CD enhancement
- [ ] (End-to-end CI tests): For SGLang, support Hicache, PD, Elatics EP, checkpoint engine tests.
- SGLang PD E2E CI: #1089
- Mooncake EP CI: #1099
- SGLang HiCache E2E CI: #1181
- vLLM 1P1D E2E CI: #1491

### Milestone 10: Performance & Benchmarks
- [ ] (Store Master Benchmark): Design and integrate a dedicated benchmark for the Mooncake store master module to evaluate throughput, latency, and scalability. #1201
- [ ] (SSD/NVMe storage benchmark) : #1388

### Milestone 11: RL Infra
- [ ] (ROLL):
- [ ] (Slime/Miles) :
- https://github.com/THUDM/slime/pull/1709
- https://github.com/radixark/miles/pull/591

-----
Thanks for being a part of the Mooncake community! Welcome to ****discuss**** and contribute!

-----
**If you have any ideas, just leave a comment below and help shape the Roadmap.**

This issue is a broad roadmap covering architecture, storage, networking, deployment, hardware, and ecosystem work rather than a single entry point or test. Start by reviewing the relevant milestone and its linked issues, then choose one unchecked item with a maintainer. Done should be defined in a separately scoped issue with specific acceptance criteria.

Written by the indexing model from the issue text.