[Enhancement] massive memory allocation in the `parsePublishMessageQueues`
- Dominant language
- Java
- Stars
- 22.6k
- Forks
- 12k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 26
Description
### Before Creating the Enhancement Request
- [x] I have confirmed that this should be classified as an enhancement rather than a bug/feature.
### Summary
During my benchmark (50K LMQ tps), the rocketmq client in my process shows massive memory allocation in the `parsePublishMessageQueues`.
Here is my profiling result about the memory allocation:
### Motivation
This multiple growths of the array list introduce a great influence on the performance under the high throughput workload.
### Describe the Solution You'd Like
Here is a simple proposal:
- do pre-allocation for the returned list of the `parsePublishMessageQueues`.
- Just return the original `List` if the namespace is empty.
### Describe Alternatives You've Considered
There are more complex solutions:
- Upgrade the interface about the `MessageQueueSelector` to use `size of the message queue` and `accessor by index to MessageQueue` instead of `List`, so that no need to create another `List`.
- Use thread-local buffer array list for the parsed `List` to avoid allocating memory for any to-be-sent message.
### Additional Context
_No response_
Contributor guide
Research direction
Locate the Java implementation of parsePublishMessageQueues and inspect how it builds the returned List, especially when the namespace is empty. Confirm the proposed allocation changes preserve the existing behavior, then verify the result with the project’s relevant tests or a high-throughput benchmark.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems, performance
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100