Use different ports (or different bind addresses) for Raft and RPC traffic
- Dominant language
- Go
- Stars
- 30.1k
- Forks
- 4.6k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 43
Description
#### Feature Description
It would be great if we can separate raft and RPC traffic, so we can prioritize Raft over RPC.
#### Use Case(s)
Recently we had a large consul outage. The reason was that the consul cluster was overloaded with queries. The main problem in this scenario was that the cluster quickly lost leadership and wasn't able to recover until we cordoned the traffic by iptable rules. I think this happened because consul is using the same network socket for Raft and RPC, so RPC flud prevented Raft from functioning correctly. The expected behavior, in this case, is to drop some RPC queries (if the server can't keep up with them) but maintain stable leadership. If we use different network sockets (or even different network interfaces) for Raft and RPC we can easily prioritize Raft over RPC by using network cgroups or socket options.
Contributor guide
Assessment
This issue has not been assessed yet.