apache / apache/pulsar-client-go
High cpu pres
- Dominant language
- Go
- Stars
- 745
- Forks
- 389
- Avg merge
- 3d 20h
- Merged PRs (30d)
- 3
Description
#### Expected behavior
The load of the cpu is less affected by the number of topic partitions
#### Actual behavior
I have a project deployed on a server, The RocketMQ (also deployed on this server) was used as a message queue. During the peak of the business, the CPU usage of the server was only 12%. When I After replacing the message queue with Tencent's TDMQ-Pulsar service, the CPU usage of my service became very high. In the past, each topic on RocketMQ I built by myself had 16 partitions, while each topic of TDMQ-Pulsar only had 4 partitions, but the CPU usage is still 4 times the previous. Even if I adjust the number of partitions for each topic to 1, the CPU usage still exceeds that of the RocketMQ version. pprof concluded that runtime.netpoll occupies too much CPU.
#### Steps to reproduce
Create a topic, set the number of partitions to 1, use producers and consumers to connect to it in the service, view and record the CPU usage. Increase the number of partitions of the topic, connect to it again, check the CPU usage and compare with the previous results.
#### System configuration
**Pulsar version**: 2.7
Contributor guide
Research direction
Start by reproducing the producer and consumer setup with a topic configured for one partition, then repeat with more partitions while recording CPU usage. Use pprof to inspect the reported runtime.netpoll activity and compare the results against the stated RocketMQ and TDMQ-Pulsar behavior. Done means identifying the cause of the excessive CPU use or documenting a confirmed client limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- distributed-systems, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100