kvcache-ai / kvcache-ai/Mooncake

[RFC] Transfer COO sparse updates as structured objects

Open
#3,953 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
6.6k
Forks
1.2k
Avg merge
3d 5h
Merged PRs (30d)
312

Description

## Summary

This RFC proposes a framework-neutral prototype for transferring COO sparse updates as Mooncake structured objects.

## Proposal

- Represent one sparse update as JSON metadata plus named indices and values buffers.
- Use the existing Mooncake Store structured-object and buffer APIs so the data path remains backed by Store and Transfer Engine transports.
- Add an address-free planner for source-fragment selection, compact tile indexes, aligned COO range reads, generation fencing, boundary filtering, coordinate rebasing, and additive scatter-add materialization.
- Keep source/target tensor identity, geometry, placement, dtype, and generation checks at the planner/facade boundary.

## Scope and limitations

This is a prototype API under mooncake-rl and mooncake-reshard. It does not yet integrate with the ScaleAligner/ROLL production update chain, does not implement cross-process physical-node deduplication, and the current Torch apply path uses host staging rather than an end-to-end device-direct zero-copy path.

The implementation and focused tests are intended to make the structured-object contract reviewable before production integration.

Contributor guide

Open the contributing guide

Research direction

The implementation is in mooncake-rl and mooncake-reshard; start by reading the existing Mooncake Store structured-object and buffer APIs plus the focused tests mentioned in the RFC. Done means a framework-neutral prototype covers the listed planner and facade checks and additive COO materialization while respecting the stated scope limits.

Written by the indexing model from the issue text.

Assessment

Tech stack
pytorch
Domain
backend-api-design, data, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.