Blizzard / Blizzard/node-rdkafka

unable to consume specific amount of messages without waiting with a setTimeout

Open
#450 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
2.2k
Forks
403
PR merge metrics
No merged PRs in 30d

Description

**Environment Information**
- OS [e.g. Mac, Arch, Windows 10]: RedHat
- Node Version [e.g. 8.2.1]: 10.6.0
- NPM Version [e.g. 5.4.2]: 6.1.0
- C++ Toolchain [e.g. Visual Studio, llvm, g++]:
- node-rdkafka version [e.g. 2.3.3]: 2.3.2

```typescript
// Non-flowing mode
consumer.connect();

consumer
.on('ready', function() {
consumer.subscribe(['librdtesting-01']);

// Read only 1000 messages
consumer.consume(1000); // <---------- This is not working...
})
.on('data', function(data) {
console.log('Message found! Contents below.');
console.log(data.value.toString());
});
```

If i'm replacing the line that I've marked above with the following, then it does work:

```typescript
setTimeout(() => consumer.consume(1000), 60000); // <-- this is working..
```

Could this be related to `node-rdkafka`? or is it something that has to do with my kafka configuration?

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the non-flowing example with node-rdkafka 2.3.2, focusing on the ready, subscribe, consume, and data event sequence shown in the issue. Trace how consume(1000) behaves when called immediately versus after setTimeout. Done means the requested number of messages can be consumed without the delay, with a regression check for the reported sequence.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, kafka, node.js
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.