apache / apache/rocketmq

[Enhancement] Avoid per-message MessageDigest lookup and length-only getBytes allocations on the proxy gRPC path

Open
#10,976 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
22.6k
Forks
12k
Avg merge
3d 1h
Merged PRs (30d)
27

Description

### Before Creating the Enhancement Request

- [x] I have confirmed that this should be classified as an enhancement rather than a bug/feature.

### Summary

Remove two per-message allocation sources on the proxy gRPC path: the `MessageDigest.getInstance("MD5")` lookup for every delivered message, and the `getBytes(UTF_8)` calls used only for length validation on every received message.

### Motivation

1. `GrpcConverter#buildSystemProperties` computes the body digest for every message delivered to a gRPC consumer, and `BinaryUtil.calculateMd5` performs a `MessageDigest.getInstance("MD5")` provider lookup plus a fresh digest instance on every call.
2. `SendMessageActivity#buildMessageProperty` (and `validateMessageGroup`) call `str.getBytes(StandardCharsets.UTF_8)` on every user-property key/value, the tag, each message key, and the message group — only to read `.length` for size validation; the byte arrays are discarded immediately. That is 2N+ transient arrays per received message (N = user property count).

### Solution

- `BinaryUtil`: keep one `MessageDigest` per thread in a `ThreadLocal` (`MessageDigest` is not thread safe) and `reset()` before each use.
- `SendMessageActivity`: add a `utf8Length(String)` helper that computes the UTF-8 encoded length without materializing the array, matching `String.getBytes(UTF_8).length` exactly, including the single-byte replacement for unpaired surrogates; replace the five call sites.

### Verification

- `BinaryUtilTest` (new): digest matches a fresh `MessageDigest` and stays stable across interleaved calls on the reused per-thread instance. `SendMessageActivityTest#testUtf8Length`: sample-by-sample equality with `getBytes(UTF_8).length` covering ASCII, CJK, supplementary (emoji), and unpaired surrogates; full class 12/12.
- Dedicated gRPC A/B on a 4-node cluster: proxy (cluster mode) plus a loopback load tool built on `rocketmq-client-java` 5.0.7 (producer 8 threads with multi-byte user properties exercising `utf8Length`, SimpleConsumer 4 threads exercising the digest path), swapping the proxy's `rocketmq-common`/`rocketmq-proxy` jars per arm; 3 interleaved trials plus 1 reversed-order control: ~9.4k send TPS / ~5.1k consume TPS, zero failures in all 8 arms; proxy young GC showed a pure positional artifact (first arm of each pair always 9, second always 10, independent of the jar — confirmed by the reversed-order control), i.e. parity after correction. No regression; the allocation saving itself is below GC-count resolution, so this is a cleanup-level optimization on the proxy hot path.

Contributor guide

Open the contributing guide

Research direction

Start with BinaryUtil#calculateMd5 and GrpcConverter#buildSystemProperties to trace the digest path, then inspect SendMessageActivity#buildMessageProperty and validateMessageGroup for the UTF-8 length checks. Run BinaryUtilTest and SendMessageActivityTest#testUtf8Length first; done means digest results remain correct and all encoded-length samples match getBytes(UTF_8).length without the discarded allocations.

Written by the indexing model from the issue text.

Assessment

Tech stack
grpc, java
Domain
api, backend, performance
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.