confluentinc / confluentinc/confluent-kafka-javascript

Memory Leak When Using opaque Parameter in produce

Open
#272 2 comments 0 reactions 0 assignees View on GitHub
bug good first issue
Dominant language
TypeScript
Stars
304
Forks
45
Avg merge
11h 47m
Merged PRs (30d)
5

Description

When calling `produce` on a `Producer` and passing any value other than `undefined` for the `opaque` parameter results in a memory leak.

#### **Steps to Reproduce**
1. Use the following code to produce messages:
```javascript
const { Producer } = require("@confluentinc/kafka-javascript");
const { once } = require("events");

const producerConfig = {
"client.id": "my-kafka-consumer",
"metadata.broker.list": "localhost:19092",
};

const topic = "test-topic";
const producer = new Producer(producerConfig);

function getRandomInt(min, max) {
const ceilMin = Math.ceil(min);
const floorMax = Math.floor(max);
return Math.floor(Math.random() * (floorMax - ceilMin + 1)) + ceilMin;
}

function startDataGeneration() {
producer.setPollInterval(10);
const intervalMs = 1;

setInterval(() => {
for (let i = 0; i < 100; i++) {
const payload = {
timestamp: new Date().toISOString(),
message: "Hello from data generator!",
random: getRandomInt(0, 999),
index: i,
};

const value = Buffer.from(JSON.stringify(payload));
const key = `key-${getRandomInt(1, 20)}`;

producer.produce(topic, null, value, key, null, { a: 1 });
}
}, intervalMs);
}

async function main() {
producer.connect();
await once(producer, "ready");
console.log("Producer is ready. Starting data generation...");
startDataGeneration();
}

main().catch((err) => {
console.error("Error in main execution:", err);
});
```
2. Run the above script and wait for at least 5 minutes.
3. Observe that memory usage continues to grow indefinitely.
4. Replace `{ a: 1 }` with `undefined` in the `opaque` parameter.
5. Notice that the memory leak does not occur when `opaque` is `undefined`.

#### **Expected Behaviour**
Passing an object to `opaque` should not cause a memory leak.

#### **Actual Behaviour**
Memory usage continues to grow indefinitely when `opaque` is set to `{ a: 1 }`, `null`, or any value other than `undefined`.

#### **Environment**
- `@confluentinc/kafka-javascript` version: v1.2.0
- Node.js version: 22.4
- OS: Arch Linux

#### **Additional Context**
- The issue persists across multiple runs.
- Removing `opaque` (or setting it to `undefined`) prevents the memory leak.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.