Blizzard / Blizzard/node-rdkafka

offsetsStore() does not work when using 'connection string' in >=2.16 version

Open
#1,021 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
2.2k
Forks
403
PR merge metrics
No merged PRs in 30d

Description

**Environment Information**
- OS: Alpine 3.17
- Node Version: 18.14.0
- NPM Version: 9.6.7
- node-rdkafka version: >=2.16

It seems that since version 2.16, which includes the new version of librdkafka, the functionality of manually committing the offset when connecting to the event hub using connection string has stopped working. Version 2.15 is working properly.

Configuration:
```
'enable.auto.commit': true,
'enable.auto.offset.store': false
```
Code:
```
const processedTopicPartition = {
topic,
partition,
offset
}
stream.consumer.offsetsStore([processedTopicPartition])
```
I checked, also, that `offset_commit_cb` is not called in this case.

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with Node.js 18.14.0, node-rdkafka >=2.16, an Event Hubs connection string, and the shown consumer configuration and offsetsStore call; compare against version 2.15. Check whether offset_commit_cb is invoked in each case. Done means manual offset storage and commit callbacks work with the newer version.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, kafka, nodejs
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.