Blizzard / Blizzard/node-rdkafka
Cooperative rebalancing consumer hangs on disconnect, rebalanceProtocol() returns NONE
- Dominant language
- JavaScript
- Stars
- 2.2k
- Forks
- 403
- PR merge metrics
- No merged PRs in 30d
Description
Hi there, I believe this is a bug in the node-rdkafka library. When we use cooperative rebalancing, on shutdown some consumers return the rebalance protocol as NONE rather than COOPERATIVE.
**Environment Information**
- OS [e.g. Mac, Arch, Windows 10]:
- Node Version [e.g. 8.2.1]: 22.22.0
- NPM Version [e.g. 5.4.2]: 10.9.4
- C++ Toolchain [e.g. Visual Studio, llvm, g++]:
- node-rdkafka version [e.g. 2.3.3]: 3.6.0
**Steps to Reproduce**
1. Create a KafkaConsumer with partition.assignment.strategy: 'cooperative-sticky' and rebalance_cb: true
2. Connect the consumer and wait for it to join a consumer group and receive partition assignments
3. Call consumer.disconnect() (or stream.destroy() on a KafkaConsumerStream)
4. Observe that the final revoke rebalance callback fires, but rebalanceProtocol() returns 'NONE' instead of 'COOPERATIVE'
5. It looks like the built-in handler calls unassign() instead of incrementalUnassign(), which throws inside librdkafka. The error is silently swallowed because isConnected() is also false at this point. disconnect() never completes.
**node-rdkafka Configuration Settings**
'partition.assignment.strategy': 'cooperative-sticky',
rebalance_cb: true,
**Additional context**
We observe this happening intermittently, when we have multiple consumers with a large number of partitions. When rebalancing happens, only some consumers return 'NONE' instead of 'COOPERATIVE'.
Contributor guide
Research direction
Start by tracing KafkaConsumer.disconnect(), KafkaConsumerStream.destroy(), and the built-in rebalance handler described in the report. Reproduce with cooperative-sticky and rebalance_cb enabled, then inspect how the final revoke handles rebalanceProtocol() returning NONE and whether unassign() is called after connection loss. Done means shutdown completes reliably and cooperative consumers use the appropriate revoke behavior without silently swallowed errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, nodejs
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100