eBay / eBay/HomeStore

Implement nuraft system_exit

Open
#852 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
29
Forks
30
Avg merge
2d 4h
Merged PRs (30d)
6

Description

When nuraft identifies an unrecoverable critical case, it will call our system_exit callback. Currently, we only print a log which is meaningless and lead to the system stuck all the time.
Here is a concrete case:
When a follower recovers from the network issue, it attempts to request pre-vote, but it is rejected by the other peers. After rejected by quorum > 20 times, it tries to reconnect to leader. However it doesn't know who is leader and goes into system_exit.
So if we can restart the raft server which will reset all the state, it will becomes self-recoverable.

Whether we can safely restart the raft server without affecting other raft group needs more investigation and tests.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at src/lib/replication/repl_dev/raft_repl_dev.h around line 459, where the NuRaft system_exit callback currently only logs, and compare it with the referenced NuRaft system_exit path. Investigate whether restarting the raft server safely resets the affected state without impacting other raft groups, then add tests demonstrating safe recovery.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.