Blizzard / Blizzard/node-rdkafka

Increase in topic partition errors out with PARTCNT and severity = 5

Open
#1,013 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
2.2k
Forks
403
PR merge metrics
No merged PRs in 30d

Description

If the number of partitions on a given topic is increased, ideally the consumer should register automatically for those partitions and start consuming the events. In our case, the consumer started consuming the events 3 days after the new partitions were added. Old events were not fetched too. Can you please provide insights on what is that config which needs to be done to automatically register the newly added partitions? We are using event.log to log the events. This is what we got in our logs :

```
{
"severity": 5,
"fac": "PARTCNT",
"context": {}
}
```

**Environment Information**
- OS [e.g. Mac, Arch, Windows 10]: Alpine Linux v3.11
- Node Version [e.g. 8.2.1]: 10.24.1
- NPM Version [e.g. 5.4.2]: 6.14.12
- C++ Toolchain [e.g. Visual Studio, llvm, g++]:
- node-rdkafka version [e.g. 2.3.3]: 2.6.1

**Steps to Reproduce**

**node-rdkafka Configuration Settings**

```
'group.id': "test",
'enable.auto.commit': false,
'fetch.message.max.bytes': 10048576,
'offset_commit_cb': (error: any) => {
if (!error) return;
},
'statistics.interval.ms': 1000,
```

**Additional context**

Contributor guide

Open the contributing guide

Research direction

The payload names no source files or tests. Start by reviewing node-rdkafka consumer configuration and the PARTCNT event.log entry against the supplied environment and settings; done means documenting the relevant configuration or confirming a reproducible defect with clear reproduction steps and logs.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, nodejs
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.