eclipse-vertx / eclipse-vertx/vert.x

HA verticle become deployed twice after split-brain partition

Open
#5,235 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
14.7k
Forks
2.1k
Avg merge
1d 10h
Merged PRs (30d)
27

Description

### Version

4.5.7

### Context

I've run three nodes Vert.x cluster with HA feature enabled and quorum size set to 2.

```
.setQuorumSize(2)
.setHAEnabled(true);
```

Only one node (let's say `Node1`) deployed a single verticle (say `SingletonVerticle`) programmatically with HA option set to true.

```
new DeploymentOptions()
.setInstances(1)
.setHa(true)
```

If `Node1` leaves a cluster because of split-brain network partition than the `SingletonVerticle` gets redeployed on one of other two nodes and undeployed on `Node1` as expected.

But when network fault is over and `Node1` joins a cluster back, `HAManager` redeploys `SingletonVerticle` on `Node1` too. Which results in two instances of `SingletonVerticle` in a cluster instead of declared count in `DeploymentOptions`.

As I can see the problem is that `HAManager` operates a local `Queue toDeployOnQuorum` of verticles waiting for deploy along with `Map clusterMap` of deployments and it doesn't check wheather verticles from the queue already deployed on other nodes.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.