confluentinc / confluentinc/confluent-kafka-javascript

Disconnected Consumers Continue To Take Up Space in Heap

Open
#369 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
TypeScript
Stars
304
Forks
45
Avg merge
11h 47m
Merged PRs (30d)
5

Description

**Environment Information**
- OS: Windows 11
- Node Version: 22.7.0
- NPM Version: 10.8.2
- C++ Toolchain: Visual Studio Code
- confluent-kafka-javascript version: 1.4.1

**Steps to Reproduce**
1. Take a snapshot of the heap usage
2. Initialize a new Kafka Consumer
3. Call the Consumer.connect() method
4. Call the Consumer.disconnect() method
5. Take a snapshot of the updated heap usage

```
for(let i = 0; i < 10; i++) {
const consumer = kafka.consumer({ kafkaJS: { groupId: `Group-${i}`, fromBeginning: false} });
await consumer.connect();
await consumer.disconnect();
}
```

Running the above snippet and taking heap snapshots between each consumer creation shows that around 1 MB of memory is retained for each disconnected consumer. This can add up over time if an application needs to spin up a large number of consumers.

Comparing the latest heap snapshot to the initial the largest size delta is in the Array constructor which added 7.2MB.

Comparing each snapshot to the previous one shows the same. The Array constructor has the largest size delta and increases at a consistent rate between snapshots.

Image

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.