apache / apache/pulsar

[improve][client]pulsar-client-io thread has high CPU usage when client idle and batch message enable

Open
#23,187 5 comments 0 reactions 0 assignees View on GitHub
type/enhancement
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Motivation

Create a pulsar client and to initialize, send a little message to topic, then the pulsar-client-io thread(just 1 thread by default) cpu usage is 14%, but client is idle, no action for it.
![image](https://github.com/user-attachments/assets/62d716fd-29dc-497b-a817-bf79de91b34d)

the root cause is that batch message sending is enable, ProducerImpl would use pulsar-client-io thread to send message per 1ms by default
![image](https://github.com/user-attachments/assets/30337978-f449-4bd0-9d29-c4c57b434bdf)

pulsar-client use EventLoopGroup as thread pool, the thread name is "pulsar-client-io",EpollEventLoopGroup is preferred.
When switch to NioEventLoopGroup, the CPU usage just reduce little to 9.6%
![image](https://github.com/user-attachments/assets/7c7bd5dd-a487-4bfe-ba0e-bcefb76fa78a)

### Solution

Use ScheduledExecutorService to send batch message to replace the EventLoopGroup. It would reduce CPU usage less than 1%
![image](https://github.com/user-attachments/assets/0f059349-c6a2-4abf-acf2-e0664b8796bf)

### Alternatives

_No response_

### Anything else?

_No response_

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start by tracing batch-message scheduling in ProducerImpl and how the pulsar-client-io EventLoopGroup currently triggers sends. Compare that path with the proposed ScheduledExecutorService approach; done means idle clients keep batching behavior while reducing pulsar-client-io CPU usage to below 1%.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.