deven96 / deven96/ahnlich

perf(replication): benchmark and reduce duplicate state-machine snapshot buffers

Open
#382 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
244
Forks
29
Avg merge
2d 41m
Merged PRs (30d)
19

Description

## Context

The durable state-machine snapshot implementation retains the current snapshot in the state machine and exposes an owned `Cursor>` to OpenRaft. For large completed snapshots, this can mean two in-process payload buffers.

This is not a correctness problem. Snapshot durability, install, restart, and compaction recovery must remain unchanged.

## Work

- Benchmark snapshot build allocation, peak RSS, and latency for realistic DB and AI snapshot sizes.
- Benchmark snapshot transfer latency and memory while a leader serves a snapshot to a learner.
- Decide from measurements whether duplicate completed-snapshot buffers are material.
- If warranted, introduce a custom OpenRaft `SnapshotData` type with a mutable receive representation for `InstallSnapshot` and a shared immutable representation for completed snapshots.
- Preserve durable local recovery, installed-snapshot recovery, and chunked transport behavior with targeted tests.

## Acceptance Criteria

- Baseline measurements are recorded.
- Any optimization is justified by those measurements.
- Restart-after-compaction and install-then-restart coverage remains intact.
- Snapshot receive/send behavior is tested for partial/chunked transfer.

Contributor guide

Open the contributing guide

Research direction

Start by locating the durable state-machine snapshot implementation and its OpenRaft SnapshotData and InstallSnapshot integration. Run baseline benchmarks for build allocation, peak RSS, latency, and leader-to-learner transfer using realistic snapshot sizes. Done means measurements justify any optimization, while restart recovery and partial or chunked snapshot transfer remain covered by targeted tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases, distributed-systems
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.