Cluster members in AKS helm deploy being mixed or polluted with members in other cluster
- Dominant language
- Go
- Stars
- 30.1k
- Forks
- 4.6k
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 39
Description
#### Overview of the Issue
We are running consul in AKS with multiple clusters each with consul 3 node servers with hundreds of members in the members list.
We noticed recently that two different AKS clusters were sharing their members list which we do not want. each cluster should only have the members from that cluster.
We did ensure we have firewalls between the two AKS clusters (both are in same network but have unique subnets) to block any consul ports but within 6 hours of tearing down consul (helm uninstall + kubectl delete ns) and re-deploying it, the other cluster's members list came back.
#### Reproduction Steps
We don't really know how to reproduce it or why it's happening.
### Consul info for both Client and Server
```
build:
prerelease =
revision = 2c56447e
version = 1.11.1
```
### Operating system and Environment details
AKS 1.27 clusters
If this is just how consul works, please guide me to where the documentation on how to prevent cross cluster data leakage - we tried blocking all the consul ports (8600 8500 8501 8502 8503 8300 8301 8302) - are we missing something?
I was not able to find anything about preventing data sharing between clusters - just how to do the opposite, i.e. sharing across DCs.
Contributor guide
Research direction
Start by reviewing the Consul 1.11.1 Helm deployments in both AKS 1.27 clusters, including the listed Consul ports and cluster membership settings. Establish a reproducible teardown and redeployment case, then verify that each cluster's members list contains only its own nodes and document the configuration or networking change that prevents cross-cluster membership.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, helm, kubernetes
- Domain
- distributed-systems, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100