Blizzard / Blizzard/node-rdkafka

Connecting to Kafka with 3.4.0 is very slow

Open
#1,128 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
2.2k
Forks
403
PR merge metrics
No merged PRs in 30d

Description

**Environment Information**
- OS [e.g. Mac, Arch, Windows 10]: Mac
- Node Version [e.g. 8.2.1]: v20
- NPM Version [e.g. 5.4.2]: v10
- C++ Toolchain [e.g. Visual Studio, llvm, g++]:
- node-rdkafka version [e.g. 2.3.3]: 3.4.0

**Steps to Reproduce**

Download the zip, unzip, ensure Kafka is running on 9020 and run:

```
npm i

// uses 3.4.0 (latest, slow)
npm run start

// uses 3.3.1 (previous, fast)
npm run start-prev
```

Run it a couple of times, you will see that the `start` command is very slow (up to 20s) and the `start-prev` is always < 50ms.

I have tried it with:
- Kafka v2 + Zookeeper v3
- Kafka v3 + Zookeeper v3

With the update from 3.3.1 to 3.4.0 we are not just experiencing a slow connection. We are in general getting suddenly random timeout errors in our tests and the delivery-report is completely broken for us. I will post my findings on #1123

Thanks!

[reproduce.zip](https://github.com/user-attachments/files/20067347/reproduce.zip)

Contributor guide

Open the contributing guide

Research direction

Start with reproduce.zip and run the npm start and npm run start-prev commands against the stated Kafka and Zookeeper versions, comparing node-rdkafka 3.4.0 with 3.3.1. Read the related findings on issue #1123; done means the slow connection, random test timeouts, and broken delivery reports are addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, kafka, nodejs
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.