linkedin / linkedin/kafka-tools

Even not working optimally?

Open
#95 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
599
Forks
145
PR merge metrics
No merged PRs in 30d

Description

The `even` mode in `balance` is the following:
```
even - This is a specialized type for clusters which have topics that have a number of partitions that are a multiple of the number of brokers. The module assures that every broker has exactly the same number of partitions for each topic.
```

Yet the code tries to SSH to my brokers?
```
[ec2-user@ip-172-31-12-216 ~]$ kafka-assigner -z $ZK_HOST -e balance --types even
[INFO] Connecting to zookeeper 172.31.9.21:2181
[INFO] Getting partition list from Zookeeper
[INFO] Closing connection to zookeeper
[INFO] Getting partition sizes via SSH for 172.31.32.31
[INFO] Getting partition sizes via SSH for 172.31.9.32
[INFO] Getting partition sizes via SSH for 172.31.16.33
[INFO] Getting partition sizes via SSH for 172.31.16.34
[INFO] Starting even partition balance
```

I would have guessed the `even` module could just connect to Zookeeper to distribute partitions evenly?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the `even` mode in `balance` and reproduce the documented command with the shown SSH logs. Compare the mode's documented partition-distribution behavior with its SSH-based partition-size collection; done means the implementation and documentation agree about whether SSH is required.

Written by the indexing model from the issue text.

Assessment

Tech stack
kafka, python
Domain
distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.