confluentinc / confluentinc/confluent-kafka-javascript
Clarification on documentation: "eachBatch’s batch size never exceeds 1.*"
- Dominant language
- TypeScript
- Stars
- 304
- Forks
- 45
- Avg merge
- 11h 47m
- Merged PRs (30d)
- 5
Description
Hi Team,
We are migrating to Javascript client from KafkaJS and using APIs under:
`const { Kafka } = require("@confluentinc/kafka-javascript").KafkaJS;`
We would like to understand what does this statement mean in migration documentation present here: [reference to doc](https://docs.confluent.io/kafka-clients/javascript/current/migration.html#javascript-migrate-from-kafkajs:~:text=eachBatch%E2%80%99s%20batch%20size%20never%20exceeds%201.)
The statement mentions:
**eachBatch’s batch size never exceeds 1.**
However, doesn't that mean eachBatch will behave just like eachMessage since size is limited to 1?
I tried to do a quick POC with this snippet:
```
await consumer.run({
eachBatch: async ({ batch, resolveOffset, heartbeat, isRunning, isStale }) => {
console.log(`[${instanceId}] Received batch with ${batch.messages.length} messages`);
console.log(`[${instanceId}] Batch info:`, {
topic: batch.topic,
partition: batch.partition,
firstOffset: batch.firstOffset(),
lastOffset: batch.lastOffset()
});
for (const message of batch.messages) {
if (!isRunning() || isStale()) break;
console.log(`[${instanceId}] Processing message:`, {
offset: message.offset,
value: message.value.toString()
});
await resolveOffset(message.offset);
//await heartbeat();
console.log(`[${instanceId}] Started`);
for (let i = 0; i < 10000000; ) {
// Simulate processing delay
i++;
}
console.log(`[${instanceId}] Done`);
}
}
});
```
and I could see in logs that batch size is coming as more than 1 frequently, proving that no such limit is not in place.
So, can you please clarify what this statement means: **eachBatch’s batch size never exceeds 1.** and if there are any plans to limit eachBatch's size to 1 since it will affect our migration.
Thanks in advance.
Contributor guide
Assessment
This issue has not been assessed yet.