Blizzard / Blizzard/node-rdkafka
Connecting to Kafka with 3.4.0 is very slow
- Dominant language
- JavaScript
- Stars
- 2.2k
- Forks
- 403
- PR merge metrics
- No merged PRs in 30d
Description
**Environment Information**
- OS [e.g. Mac, Arch, Windows 10]: Mac
- Node Version [e.g. 8.2.1]: v20
- NPM Version [e.g. 5.4.2]: v10
- C++ Toolchain [e.g. Visual Studio, llvm, g++]:
- node-rdkafka version [e.g. 2.3.3]: 3.4.0
**Steps to Reproduce**
Download the zip, unzip, ensure Kafka is running on 9020 and run:
```
npm i
// uses 3.4.0 (latest, slow)
npm run start
// uses 3.3.1 (previous, fast)
npm run start-prev
```
Run it a couple of times, you will see that the `start` command is very slow (up to 20s) and the `start-prev` is always < 50ms.
I have tried it with:
- Kafka v2 + Zookeeper v3
- Kafka v3 + Zookeeper v3
With the update from 3.3.1 to 3.4.0 we are not just experiencing a slow connection. We are in general getting suddenly random timeout errors in our tests and the delivery-report is completely broken for us. I will post my findings on #1123
Thanks!
[reproduce.zip](https://github.com/user-attachments/files/20067347/reproduce.zip)
Contributor guide
Research direction
Start with reproduce.zip and run the npm start and npm run start-prev commands against the stated Kafka and Zookeeper versions, comparing node-rdkafka 3.4.0 with 3.3.1. Read the related findings on issue #1123; done means the slow connection, random test timeouts, and broken delivery reports are addressed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, kafka, nodejs
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100