confluentinc / confluentinc/confluent-kafka-javascript

Clarification on documentation: "eachBatch’s batch size never exceeds 1.*"

Open
#350 7 comments 0 reactions 0 assignees View on GitHub
documentation question
Dominant language
TypeScript
Stars
304
Forks
45
Avg merge
11h 47m
Merged PRs (30d)
5

Description

Hi Team,

We are migrating to Javascript client from KafkaJS and using APIs under:
`const { Kafka } = require("@confluentinc/kafka-javascript").KafkaJS;`

We would like to understand what does this statement mean in migration documentation present here: [reference to doc](https://docs.confluent.io/kafka-clients/javascript/current/migration.html#javascript-migrate-from-kafkajs:~:text=eachBatch%E2%80%99s%20batch%20size%20never%20exceeds%201.)

The statement mentions:
**eachBatch’s batch size never exceeds 1.**
However, doesn't that mean eachBatch will behave just like eachMessage since size is limited to 1?

I tried to do a quick POC with this snippet:

```
await consumer.run({
eachBatch: async ({ batch, resolveOffset, heartbeat, isRunning, isStale }) => {
console.log(`[${instanceId}] Received batch with ${batch.messages.length} messages`);
console.log(`[${instanceId}] Batch info:`, {
topic: batch.topic,
partition: batch.partition,
firstOffset: batch.firstOffset(),
lastOffset: batch.lastOffset()
});

for (const message of batch.messages) {
if (!isRunning() || isStale()) break;

console.log(`[${instanceId}] Processing message:`, {
offset: message.offset,
value: message.value.toString()
});

await resolveOffset(message.offset);
//await heartbeat();
console.log(`[${instanceId}] Started`);
for (let i = 0; i < 10000000; ) {
// Simulate processing delay
i++;
}
console.log(`[${instanceId}] Done`);

}
}
});
```

and I could see in logs that batch size is coming as more than 1 frequently, proving that no such limit is not in place.

So, can you please clarify what this statement means: **eachBatch’s batch size never exceeds 1.** and if there are any plans to limit eachBatch's size to 1 since it will affect our migration.

Thanks in advance.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.