Blizzard / Blizzard/node-rdkafka
consumer.connect opens thread but disconnect does not close it
- Dominant language
- JavaScript
- Stars
- 2.2k
- Forks
- 403
- PR merge metrics
- No merged PRs in 30d
Description
Currently when connecting a consumer it starts a thread for that connection. However if you disconnect the consumer the thread remains open. There also seems to be no method on the consumer to close its respective thread. This leaves you effectively with a limited number of consumers to spawn even if older ones are beeing disconnected after use (limited by the UV_THREADPOOL_SIZE which in turn is limited to 1024).
Am I missing something here because this seems like unwanted behaviour.
**Environment Information**
- OS [Win10-WSL-Ubuntu20.04]:
- Node Version [e.g. 18]:
- NPM Version [e.g. 8]:
- node-rdkafka version [2.13.0]:
**Steps to Reproduce**
1. Start a consumer and connect it
2. use ```ps aux``` to retrieve node PID
3. use ```ps -T -p [PID]``` to see open threads and find rdk:main for example
4. disconnect the consumer and see how the threads stay open
5. repeat the preceding steps to find new threads beeing spawned but no old ones beeing killed off again.
Contributor guide
Research direction
Start at the consumer.connect and disconnect entry points and reproduce the issue using the listed ps aux and ps -T commands. Trace how the rdk:main thread is created and handled during disconnect. Done means disconnecting a consumer no longer leaves its connection thread open, confirmed by the reproduction steps.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, kafka, nodejs
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100