[Enhancement] Optimize message properties encode/decode to reduce allocation
- Dominant language
- Java
- Stars
- 22.6k
- Forks
- 12k
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 27
Description
### Before Creating the Enhancement Request
- [x] I have confirmed that this should be classified as an enhancement rather than a bug/feature.
### Summary
Replace per-message `HashMap` allocation in the properties encode/decode hot path with a compact `FlatPropertiesMap` backed by a flat `Object[]` array. Add ThreadLocal reuse for the map, StringBuilder, and char[] buffers. Intern high-frequency property keys and values to eliminate redundant String allocation.
**Scope**: 8 files in `common/message`
### Motivation
Heap dump analysis reveals:
- **`HashMap$Node[]`** occupies **21.0%** of live heap — every message creates multiple HashMaps for properties parsing.
- **`String`** occupies **8.1%** with only **21,873 distinct values out of 100,000 sampled** — `"true"` duplicated 7,142 times, `"UNIQ_KEY"` 4,059 times, etc.
- `byte[]` occupies **29.3%** — partly from backing arrays of duplicated small Strings.
The `string2messageProperties` / `messageProperties2String` path is the single largest allocation hotspot on the broker send path.
### Describe the Solution You'd Like
1. **`FlatPropertiesMap`** (new class): A compact `Map` implementation using a flat `Object[]` (key-value pairs). Supports `reset()` for ThreadLocal reuse, `computeEncodedLength()`, and `encodeTo(ByteBuffer)` for zero-copy serialization.
2. **`MessageDecoder`**: New `bytes2messageProperties(ByteBuffer)` method that parses properties byte-by-byte without `new String()` + `split()`. Uses `REUSABLE_PROPS_MAP` ThreadLocal for map reuse and `REUSABLE_SB` ThreadLocal for StringBuilder reuse.
3. **`MessageConst`**: Add `STRING_INTERN_BY_LEN` — length-bucketed arrays for key interning. Known property keys (e.g., `KEYS`, `TAGS`, `WAIT`, `UNIQ_KEY`) are resolved to static constants by length + first-char matching.
4. **`MessageDecoder`**: Add `VALUE_INTERN_BY_LEN` for common value interning (`"true"`, `"false"`, `"DefaultRegion"`, etc.).
5. **`MessageClientIDSetter`**: ThreadLocal `char[]` buffer for `createUniqID` to avoid per-call allocation.
6. **`MessageVersion`**: Replace `values()` iteration with direct `if-else` lookup.
### Describe Alternatives You've Considered
- **`String.intern()` (JVM native)**: Risks polluting the JVM String table and has unpredictable GC interaction; length-bucketed array interning is deterministic and bounded.
- **Keep `HashMap` but pool it**: Still pays the overhead of `HashMap.put()` internal Node allocation; `FlatPropertiesMap` avoids this entirely for the typical 8–12 property case.
- **Protocol-level binary properties** (like Pulsar): Would require wire format changes; out of scope for a backward-compatible enhancement.
### Additional Context
- `FlatPropertiesMap` is only used on the broker-internal decode→encode path within a single thread; it does not escape to user-facing APIs.
- The send-path `string2messageProperties` in `MessageDecoder` returns `HashMap` (not `FlatPropertiesMap`) to maintain compatibility with downstream code that expects `HashMap`.
- JFR-measured bytes/msg reduction: **-42.7%** from this change alone (from 4,594 to 2,631 bytes/msg).
Contributor guide
Research direction
Start with the string2messageProperties and messageProperties2String paths in common/message, then read MessageDecoder, MessageConst, MessageClientIDSetter, and MessageVersion. Compare the proposed FlatPropertiesMap and ThreadLocal changes with existing compatibility requirements; done means the eight-file broker send-path scope is implemented without breaking HashMap consumers and allocation measurements improve.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100