confluentinc / confluentinc/confluent-kafka-javascript

CPU spike when using eachMessage

Open
#326 5 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
TypeScript
Stars
304
Forks
45
Avg merge
11h 47m
Merged PRs (30d)
5

Description

**Environment Information**
- OS : node-alpine image running in ECS fargate
- Node Version: 20
- NPM Version: 20
- C++ Toolchain [e.g. Visual Studio, llvm, g++]: Visual Studio Code
- confluent-kafka-javascript version [e.g. 2.3.3]: latest

**Steps to Reproduce**
We are currently using the confluent-kafka-js library to consume messages from an AWS MSK (Provisioned) cluster. Our implementation relies on the eachMessage handler, with the expectation that messages are consumed and processed one at a time, in order. While the behavior aligns with our expectations in terms of message ordering and processing, we are observing significant CPU spikes when consumption begins, especially when the topic contains a large number of messages.

We are trying to determine whether the library is pre-fetching or consuming messages in the background, which might be contributing to the spike. If that’s the case, is there a way to control or throttle message consumption to mitigate the CPU usage?

I have attached the sample code.

[SampleKafkaJS.txt](https://github.com/user-attachments/files/20437598/SampleKafkaJS.txt)

**confluent-kafka-javascript Configuration Settings**
Basic settings. No additional setting provided.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.