hyperledger / hyperledger/fabric
single orderer network : if the orderer is restarted, it becomes a follower and is not starting an election for the orderer to become a leader
- Dominant language
- Go
- Stars
- 16.7k
- Forks
- 9.1k
- Avg merge
- 5h 38m
- Merged PRs (30d)
- 26
Description
### Description
SETUP
1. Hyperledger Fabric version: 2.2
2. Consensus: RAFT
3. Blockchain network: 1
4. Organizations:
- 2 [each org is on a different Azure kubernetes cluster]
- Each Org is on a separate channel (so 2 channels, these are named cefiuschannel & ibnpublic)
- Each Org has 1 peer each
5. Orderers: 1
ISSUE ON HAND
1. We were trying to update the blockchain certificate before its expiry for which we did the following steps:
a. We increased the validity of the new Orderer certificates from 1 year to 3 years.
b. Restarted the Certificate Authority (CA)
c. Restarted the Orderer POD
Post this we expected the following to happen:
- Orderer should have successfully restarted and started an election which would make it the Leader (as there is only 1 Orderer).
- Post this the certificates update commands should've been successfully executed.
However, we are experiencing the following:
- Orderer does restart successfully BUT it immediately becomes a Follower without even attempting an election
- The logs show the following:
2023-04-10 05:29:46.377 UTC 076a INFO [orderer.common.cluster] Configure -> Entering, channel: cefiuschannel, nodes: []
2023-04-10 05:29:46.377 UTC 076b INFO [orderer.common.cluster] Configure -> Exiting
2023-04-10 05:29:46.377 UTC 076c DEBU [orderer.consensus.etcdraft] start -> Starting raft node: #peers: 1 channel=cefiuschannel node=1
2023-04-10 05:29:46.377 UTC 076d INFO [orderer.consensus.etcdraft] start -> Starting raft node to join an existing channel channel=cefiuschannel node=1
2023-04-10 05:29:46.377 UTC 076e INFO [orderer.consensus.etcdraft] becomeFollower -> 1 became follower at term 0 channel=cefiuschannel node=1
2023-04-10 05:29:46.377 UTC 076f INFO [orderer.consensus.etcdraft] newRaft -> newRaft 1 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0] channel=cefiuschannel node=1
2023-04-10 05:29:46.377 UTC 0770 INFO [orderer.consensus.etcdraft] becomeFollower -> 1 became follower at term 1 channel=cefiuschannel node=1
2023-04-10 05:29:46.377 UTC 0771 INFO [orderer.consensus.etcdraft] Start -> Starting Raft node channel=ibnpublic node=1
2023-04-10 05:29:46.377 UTC 0772 INFO [orderer.common.cluster] Configure -> Entering, channel: ibnpublic, nodes: []
2023-04-10 05:29:46.377 UTC 0773 INFO [orderer.common.cluster] Configure -> Exiting
2023-04-10 05:29:46.377 UTC 0774 DEBU [orderer.consensus.etcdraft] start -> Starting raft node: #peers: 1 channel=ibnpublic node=1
2023-04-10 05:29:46.378 UTC 0775 INFO [orderer.consensus.etcdraft] start -> Starting raft node to join an existing channel channel=ibnpublic node=1
2023-04-10 05:29:46.378 UTC 0776 INFO [orderer.consensus.etcdraft] becomeFollower -> 1 became follower at term 0 channel=ibnpublic node=1
2023-04-10 05:29:46.378 UTC 0777 INFO [orderer.consensus.etcdraft] newRaft -> newRaft 1 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0] channel=ibnpublic node=1
2023-04-10 05:29:46.378 UTC 0778 INFO [orderer.consensus.etcdraft] becomeFollower -> 1 became follower at term 1 channel=ibnpublic node=1
2023-04-10 05:29:46.428 UTC 0779 INFO [orderer.common.server] Main -> Starting orderer:
### Steps to reproduce
1. Bring up a standard HLF network (version 2.2 with a system channel) with a single orderer on a kubernetes cluster.
2. Once the network is up, restart the orderer pod by deleting it or by restarting through the deployments.
3. The orderer becomes a follower in all the channels and fails to start an election.
Attaching the orderer log after the restart.
[orderer0-deployment-7b67b8d496-ncccs.log](https://github.com/hyperledger/fabric/files/11197714/orderer0-deployment-7b67b8d496-ncccs.log)
Contributor guide
Assessment
This issue has not been assessed yet.