apache / apache/rocketmq

[Enhancement] Optimize message properties encode/decode to reduce allocation

Open
#10,442 2 comments 0 reactions 0 assignees View on GitHub
type/enhancement
Dominant language
Java
Stars
22.6k
Forks
12k
Avg merge
3d 1h
Merged PRs (30d)
27

Description

### Before Creating the Enhancement Request

- [x] I have confirmed that this should be classified as an enhancement rather than a bug/feature.

### Summary

Replace per-message `HashMap` allocation in the properties encode/decode hot path with a compact `FlatPropertiesMap` backed by a flat `Object[]` array. Add ThreadLocal reuse for the map, StringBuilder, and char[] buffers. Intern high-frequency property keys and values to eliminate redundant String allocation.

**Scope**: 8 files in `common/message`

### Motivation

Heap dump analysis reveals:
- **`HashMap$Node[]`** occupies **21.0%** of live heap — every message creates multiple HashMaps for properties parsing.
- **`String`** occupies **8.1%** with only **21,873 distinct values out of 100,000 sampled** — `"true"` duplicated 7,142 times, `"UNIQ_KEY"` 4,059 times, etc.
- `byte[]` occupies **29.3%** — partly from backing arrays of duplicated small Strings.

The `string2messageProperties` / `messageProperties2String` path is the single largest allocation hotspot on the broker send path.

### Describe the Solution You'd Like

1. **`FlatPropertiesMap`** (new class): A compact `Map` implementation using a flat `Object[]` (key-value pairs). Supports `reset()` for ThreadLocal reuse, `computeEncodedLength()`, and `encodeTo(ByteBuffer)` for zero-copy serialization.
2. **`MessageDecoder`**: New `bytes2messageProperties(ByteBuffer)` method that parses properties byte-by-byte without `new String()` + `split()`. Uses `REUSABLE_PROPS_MAP` ThreadLocal for map reuse and `REUSABLE_SB` ThreadLocal for StringBuilder reuse.
3. **`MessageConst`**: Add `STRING_INTERN_BY_LEN` — length-bucketed arrays for key interning. Known property keys (e.g., `KEYS`, `TAGS`, `WAIT`, `UNIQ_KEY`) are resolved to static constants by length + first-char matching.
4. **`MessageDecoder`**: Add `VALUE_INTERN_BY_LEN` for common value interning (`"true"`, `"false"`, `"DefaultRegion"`, etc.).
5. **`MessageClientIDSetter`**: ThreadLocal `char[]` buffer for `createUniqID` to avoid per-call allocation.
6. **`MessageVersion`**: Replace `values()` iteration with direct `if-else` lookup.

### Describe Alternatives You've Considered

- **`String.intern()` (JVM native)**: Risks polluting the JVM String table and has unpredictable GC interaction; length-bucketed array interning is deterministic and bounded.
- **Keep `HashMap` but pool it**: Still pays the overhead of `HashMap.put()` internal Node allocation; `FlatPropertiesMap` avoids this entirely for the typical 8–12 property case.
- **Protocol-level binary properties** (like Pulsar): Would require wire format changes; out of scope for a backward-compatible enhancement.

### Additional Context

- `FlatPropertiesMap` is only used on the broker-internal decode→encode path within a single thread; it does not escape to user-facing APIs.
- The send-path `string2messageProperties` in `MessageDecoder` returns `HashMap` (not `FlatPropertiesMap`) to maintain compatibility with downstream code that expects `HashMap`.
- JFR-measured bytes/msg reduction: **-42.7%** from this change alone (from 4,594 to 2,631 bytes/msg).

Contributor guide

Open the contributing guide

Research direction

Start with the string2messageProperties and messageProperties2String paths in common/message, then read MessageDecoder, MessageConst, MessageClientIDSetter, and MessageVersion. Compare the proposed FlatPropertiesMap and ThreadLocal changes with existing compatibility requirements; done means the eight-file broker send-path scope is implemented without breaking HashMap consumers and allocation measurements improve.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.