Blizzard / Blizzard/node-rdkafka

consumer.connect opens thread but disconnect does not close it

Open
#971 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
2.2k
Forks
403
PR merge metrics
No merged PRs in 30d

Description

Currently when connecting a consumer it starts a thread for that connection. However if you disconnect the consumer the thread remains open. There also seems to be no method on the consumer to close its respective thread. This leaves you effectively with a limited number of consumers to spawn even if older ones are beeing disconnected after use (limited by the UV_THREADPOOL_SIZE which in turn is limited to 1024).

Am I missing something here because this seems like unwanted behaviour.

**Environment Information**
- OS [Win10-WSL-Ubuntu20.04]:
- Node Version [e.g. 18]:
- NPM Version [e.g. 8]:
- node-rdkafka version [2.13.0]:

**Steps to Reproduce**

1. Start a consumer and connect it
2. use ```ps aux``` to retrieve node PID
3. use ```ps -T -p [PID]``` to see open threads and find rdk:main for example
4. disconnect the consumer and see how the threads stay open
5. repeat the preceding steps to find new threads beeing spawned but no old ones beeing killed off again.

Contributor guide

Open the contributing guide

Research direction

Start at the consumer.connect and disconnect entry points and reproduce the issue using the listed ps aux and ps -T commands. Trace how the rdk:main thread is created and handled during disconnect. Done means disconnecting a consumer no longer leaves its connection thread open, confirmed by the reproduction steps.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, kafka, nodejs
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.