eBay / eBay/HomeStore

[Resync] Baseline resync: old member rejoin

Open
#262 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Replication Resync
Dominant language
C++
Stars
29
Forks
30
Avg merge
2d 4h
Merged PRs (30d)
6

Description

old member re-joining after a crash or shutdown for sometime which makes its last committed lsn not in scope of existing quorum, in other words, can't do incremental resync.

Phase-1:
We can still use (#261 ) to resync to this member. Old member needs to format itself as a fresh booted member if it detects (needs some API between leader and follower to get this knowledge) it can't be served as an incremental resync.

Phase-2 (Nice to have?):
Get the last snapshot and get its log index saved in its snapshot and try to get a diff entries from the index btree on leader side:

  • Query the btree to return all the entries whose LSN is larger than the LSN mapped to log index saved in this new member’s last snapshot. Need a full scan of the btree entries. Currently there is no api to do range query based on LSN.

  • Pack scanned entries (only keys are needed) with remaining log entries to the remote side and have the remote side to fetch data based on key.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing issue #261 and the current incremental-resync implementation; the payload names no files or tests. Clarify whether Phase 1 is the target, then verify the behavior for an out-of-scope LSN; Phase 2 requires a separate design for LSN-based range queries and snapshot log-index handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.