Blizzard / Blizzard/node-rdkafka

Odd Memory Usage

Open
#1,135 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
2.2k
Forks
403
PR merge metrics
No merged PRs in 30d

Description

Hi!

I've been using this library for a while and noticed an unusual behavior regarding memory usage. When my consumer receives a lot of messages, the Node.js application's memory usage naturally increases. However, I've observed that while heap usage decreases after processing the messages, the .rss (resident set size) memory remains significantly higher than expected.

Here is an example of what I'm experiencing:

- **Initial Memory Usage:**
RSS: 200 MB
Heap: 50 MB
- **During Message Processing:**
RSS: 1.5 GB
Heap: 500 MB
- **After Processing:**
RSS: 1 GB
Heap: 60 MB

Is this the expected behaviour?

Thanks!

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are identified in the issue. Start by reproducing the reported memory pattern in the Node.js Kafka consumer and inspect the relevant node-rdkafka and librdkafka behavior; done means establishing whether the RSS retention is expected and documenting or isolating a reproducible defect.

Written by the indexing model from the issue text.

Assessment

Tech stack
kafka, nodejs
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.