kvcache-ai / kvcache-ai/Mooncake

[SegmentPool] Refactor Memory and CXL management into SegmentPool

Open
#3,363 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
6.6k
Forks
1.2k
Avg merge
3d 5h
Merged PRs (30d)
312

Description

Parent RFC: #3360
Depends on: #3362

## Goal

Turn the remaining Memory/CXL segment manager into the region-oriented `SegmentPool` boundary.

## Scope

- Refactor or rename the post-LocalSSD `SegmentManager` into `SegmentPool`.
- Define a mounted pool element as a client-mounted, addressable allocation region.
- Keep Memory and CXL catalog, indexes, allocator ownership, status, and two-phase unmount inside the pool.
- Introduce stateful Memory/CXL region drivers that own physical allocator resources.
- Extract pointer-based placement indexing and concrete replica allocation while preserving all existing policy behavior.
- Separate allocator and pool snapshot codecs while preserving the current wire shape.
- Update internal MasterService call sites and tests while retaining external RPC contracts.

## Out of scope

- Moving NoF state into the pool; that is tracked separately.
- Changing placement-policy behavior, segment-name grouping semantics, locality ordering, or CXL replica representation.
- Generalizing promotion targets.

## Implementation stack

Mechanisms land before the final caller switch:

| Order | PR | Review focus |
| ---: | --- | --- |
| 1 | #3703 | Region-driver resource ownership, import, and rollback |
| 2 | #3704 | Placement pointer stability, policy feedback, and hot-path allocation |
| 3 | #3705 | SegmentPool catalog/index invariants and lifecycle transactions |
| 4 | #3706 | Snapshot validation, allocator adoption, and fork safety |
| 5 | #3707 | MasterService/NoF call-site cutover and legacy-path deletion |

Each PR is based on the preceding PR. Merge in order; do not retarget later layers to `main` until their dependency has landed.

## Acceptance criteria

- `SegmentPool` is the sole owner of Memory/CXL mounted-region and allocator state.
- CXL remains represented as a MEMORY replica and keeps its existing global allocator behavior.
- Same-name segment grouping and existing mount/remount/unmount error behavior are preserved.
- Snapshot wire shape and LocalSSD/NoF ownership boundaries are preserved.
- Release PutStart comparisons remain within 3% single-thread QPS, 5% multi-thread QPS, and 10% p99 regression thresholds.
- Focused driver, placement, SegmentPool, snapshot, NoF mapping, LocalSSD, offload, promotion, and HA tests pass.

## Contribution notes

Review the stack in the listed order. Keep feedback on resource ownership, placement execution, pool lifecycle, snapshot boundaries, and caller cutover attached to the corresponding layer. NoF catalog/resource migration remains in #3364.

Contributor guide

Open the contributing guide

Research direction

Read the parent RFC #3360 and dependency #3362, then follow the implementation stack from #3703 through #3707 in order. Trace SegmentManager, SegmentPool, the Memory/CXL region drivers, and MasterService call sites; the work is done when ownership, lifecycle, snapshot, and external RPC boundaries remain correct and the listed focused tests and performance thresholds pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, distributed-systems, infrastructure, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.