perf(replication): benchmark and reduce duplicate state-machine snapshot buffers
- Dominant language
- Rust
- Stars
- 244
- Forks
- 29
- Avg merge
- 2d 41m
- Merged PRs (30d)
- 19
Description
## Context
The durable state-machine snapshot implementation retains the current snapshot in the state machine and exposes an owned `Cursor>` to OpenRaft. For large completed snapshots, this can mean two in-process payload buffers.
This is not a correctness problem. Snapshot durability, install, restart, and compaction recovery must remain unchanged.
## Work
- Benchmark snapshot build allocation, peak RSS, and latency for realistic DB and AI snapshot sizes.
- Benchmark snapshot transfer latency and memory while a leader serves a snapshot to a learner.
- Decide from measurements whether duplicate completed-snapshot buffers are material.
- If warranted, introduce a custom OpenRaft `SnapshotData` type with a mutable receive representation for `InstallSnapshot` and a shared immutable representation for completed snapshots.
- Preserve durable local recovery, installed-snapshot recovery, and chunked transport behavior with targeted tests.
## Acceptance Criteria
- Baseline measurements are recorded.
- Any optimization is justified by those measurements.
- Restart-after-compaction and install-then-restart coverage remains intact.
- Snapshot receive/send behavior is tested for partial/chunked transfer.
Contributor guide
Research direction
Start by locating the durable state-machine snapshot implementation and its OpenRaft SnapshotData and InstallSnapshot integration. Run baseline benchmarks for build allocation, peak RSS, latency, and leader-to-learner transfer using realistic snapshot sizes. Done means measurements justify any optimization, while restart recovery and partial or chunked snapshot transfer remain covered by targeted tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases, distributed-systems
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100