eclipse-vertx / eclipse-vertx/vert.x
HA verticle become deployed twice after split-brain partition
- Dominant language
- Java
- Stars
- 14.7k
- Forks
- 2.1k
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 27
Description
### Version
4.5.7
### Context
I've run three nodes Vert.x cluster with HA feature enabled and quorum size set to 2.
```
.setQuorumSize(2)
.setHAEnabled(true);
```
Only one node (let's say `Node1`) deployed a single verticle (say `SingletonVerticle`) programmatically with HA option set to true.
```
new DeploymentOptions()
.setInstances(1)
.setHa(true)
```
If `Node1` leaves a cluster because of split-brain network partition than the `SingletonVerticle` gets redeployed on one of other two nodes and undeployed on `Node1` as expected.
But when network fault is over and `Node1` joins a cluster back, `HAManager` redeploys `SingletonVerticle` on `Node1` too. Which results in two instances of `SingletonVerticle` in a cluster instead of declared count in `DeploymentOptions`.
As I can see the problem is that `HAManager` operates a local `Queue toDeployOnQuorum` of verticles waiting for deploy along with `Map clusterMap` of deployments and it doesn't check wheather verticles from the queue already deployed on other nodes.
Contributor guide
Assessment
This issue has not been assessed yet.