Blizzard / Blizzard/node-rdkafka
offsetsStore() does not work when using 'connection string' in >=2.16 version
- Dominant language
- JavaScript
- Stars
- 2.2k
- Forks
- 403
- PR merge metrics
- No merged PRs in 30d
Description
**Environment Information**
- OS: Alpine 3.17
- Node Version: 18.14.0
- NPM Version: 9.6.7
- node-rdkafka version: >=2.16
It seems that since version 2.16, which includes the new version of librdkafka, the functionality of manually committing the offset when connecting to the event hub using connection string has stopped working. Version 2.15 is working properly.
Configuration:
```
'enable.auto.commit': true,
'enable.auto.offset.store': false
```
Code:
```
const processedTopicPartition = {
topic,
partition,
offset
}
stream.consumer.offsetsStore([processedTopicPartition])
```
I checked, also, that `offset_commit_cb` is not called in this case.
Contributor guide
Research direction
Reproduce the issue with Node.js 18.14.0, node-rdkafka >=2.16, an Event Hubs connection string, and the shown consumer configuration and offsetsStore call; compare against version 2.15. Check whether offset_commit_cb is invoked in each case. Done means manual offset storage and commit callbacks work with the newer version.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, kafka, nodejs
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100