Azure / Azure/terraform

Unable to start Kafka server on Azure VMs using Terraform

Open
#228 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
HCL
Stars
925
Forks
930
Avg merge
4d 15h
Merged PRs (30d)
5

Description

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Where would you like this feature added?

Other/Unknown

### Is this feature ask due to a problem that you are encountering?

While working to deploy Kafka cluster on Azure VMs using Terraform and Ansible, I am facing issues while creating a topic, I have checked and found that all three Zookeeper servers are running in cluster but one of the Kafka servers is not getting started in cluster.

VMs used: Dpsv5 series

OS: Ubuntu 20.04

Architecture: amd64 and arm64

The error which I am getting is:

```
[2023-05-30 08:35:45,623] INFO [Controller id=1, targetBrokerId=1] Node 1 disconnected. (org.apache.kafka.clients.NetworkClient)

[2023-05-30 08:35:45,623] WARN [Controller id=1, targetBrokerId=1] Connection to node 1 (myvm.internal.cloudapp.net/10.1.0.9:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

[2023-05-30 08:35:45,623] INFO [Controller id=1, targetBrokerId=1] Client requested connection close from node 1 (org.apache.kafka.clients.NetworkClient)
```

The code written to create the resources through terraform in main.tf file:

[main.txt](https://github.com/Azure/terraform/files/11612111/main.txt)
[providers.txt](https://github.com/Azure/terraform/files/11630935/providers.txt)
[variables.txt](https://github.com/Azure/terraform/files/11630937/variables.txt)
[zookeeper_cluster.txt](https://github.com/Azure/terraform/files/11630938/zookeeper_cluster.txt)
[kafka_cluster.txt](https://github.com/Azure/terraform/files/11630942/kafka_cluster.txt)

Steps followed to deploy the kafka cluster.

Run the following commands to create resources through terraform:

```

terraform init

terraform apply

```

To know the ip of kafka servers:

```

cat /tmp/inventery

```

To install the Zookeeper and the required dependencies and start zookeeper server:

```

ansible-playbook zookeeper_cluster.yaml -i /tmp/inventory

```

To install the Kafka and the required dependencies:

```

ansible-playbook kafka_cluster.yaml -i /tmp/inventory

```

To start kafka on all three server first do ssh on all three machines on which kafka is installed and run below command:

```

bin/kafka-server-start.sh config/server.properties

```

However, When I have created virtual machines manually(not through Terraform) and installed zookeeper and kafka in them with the same configuration and tried running Kafka cluster and it is running fine.

It will be really helpful if you share pointers on the above mentioned issue.

### Describe potential solutions.

When I have created virtual machines manually(not through Terraform) and installed zookeeper and kafka in them with the same configuration and tried running Kafka cluster and it is running fine.

### Anything else?

_No response_

Contributor guide

No contributing guide indexed for this repository

Research direction

Review the attached main.txt, providers.txt, variables.txt, zookeeper_cluster.txt, and kafka_cluster.txt files, then compare the Terraform/Ansible deployment with the manually created VMs. Reproduce the deployment with terraform init, terraform apply, and the listed ansible-playbook commands, then inspect the Kafka startup output around the broker connection failure. Done means identifying the configuration or infrastructure difference that prevents one broker from joining the cluster.

Written by the indexing model from the issue text.

Assessment

Tech stack
ansible, azure, kafka, terraform, ubuntu
Domain
cloud, distributed-systems, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.