Peer xxx failed to catch up
- Dominant language
- C++
- Stars
- 4.2k
- Forks
- 923
- PR merge metrics
- No merged PRs in 30d
Description
follower重启后有一定概率出现这个错误,然后就加不进去
```bash
node lgraph:192.168.1.111:19091:0 can't do pre_vote as it is not in 192.168.1.111:19092:0,192.168.1.111:19093:0
Failed to join group: [E10003][C317683200A33A3ECD0844F9EC364D80][E10003]Peer 192.168.1.111:19091:0 failed to catch up
```
leader端消息:
```bash
Receive AddPeerRequest to lgraph:192.168.1.111:19092:0 from 192.168.1.111:60788, adding 192.168.1.111:19091:0
Fail to issue RPC to 192.168.1.111:19091:0 _consecutive_error_times=1, [E112]Not connected to 192.168.1.111:19091 yet, server_id=128 [R1][E112]Not connected to 192.168.1.111:19091 yet, server_id=128 [R2][E112]Not connected to 192.168.1.111:19091 yet, server_id=128 [R3][E112]Not connected to 192.168.1.111:19091 yet, server_id=128
Fail to issue RPC to 192.168.1.111:19091:0 _consecutive_error_times=11, [E112]Not connected to 192.168.1.111:19091 yet, server_id=128 [R1][E112]Not connected to 192.168.1.111:19091 yet, server_id=128 [R2][E112]Not connected to 192.168.1.111:19091 yet, server_id=128 [R3][E112]Not connected to 192.168.1.111:19091 yet, server_id=128
Fail to issue RPC to 192.168.1.111:19091:0 _consecutive_error_times=21, [E112]Not connected to 192.168.1.111:19091 yet, server_id=128 [R1][E112]Not connected to 192.168.1.111:19091 yet, server_id=128 [R2][E112]Not connected to 192.168.1.111:19091 yet, server_id=128 [R3][E112]Not connected to 192.168.1.111:19091 yet, server_id=128
Fail to issue RPC to 192.168.1.111:19091:0 _consecutive_error_times=31, [E112]Not connected to 192.168.1.111:19091 yet, server_id=128 [R1][E112]Not connected to 192.168.1.111:19091 yet, server_id=128 [R2][E112]Not connected to 192.168.1.111:19091 yet, server_id=128 [R3][E112]Not connected to 192.168.1.111:19091 yet, server_id=128
Node lgraph:192.168.1.111:19092:0 fail to catch up peer 192.168.1.111:19091:0 when trying to change peers from 192.168.1.111:19092:0,192.168.1.111:19093:0 to 192.168.1.111:19091:0,192.168.1.111:19092:0,192.168.1.111:19093:0
```
这可能是什么问题引起的?
目前follower进程退出时会调braft::cli::remove_peer()把自己从group里去掉,这样处理有没有问题?
Contributor guide
No contributing guide indexed for this repository
Research direction
The report provides no source file or test; begin with the follower-restart path and the braft::cli::remove_peer() call, then trace the logged pre_vote, AddPeerRequest, and catch-up failures. Reproduce the intermittent restart case if possible. Done means the cause is isolated and the safety of removing a follower during shutdown is documented or corrected, with a regression test if an existing test location is found.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100