[Enhancement] Optimize RocksDB ConsumeQueue iterator for sequential reads
- Dominant language
- Java
- Stars
- 22.6k
- Forks
- 12k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 26
Description
### Before Creating the Enhancement Request
- [x] I have confirmed that this should be classified as an enhancement rather than a bug/feature.
### Summary
Currently, when RocketMQ uses RocksDB-based ConsumeQueue, the default read path is controlled by `iteratorWhenUseRocksdbConsumeQueue`, which is enabled by default.
However, this iterator is a RocketMQ-level reusable iterator. Internally, it still calls `RocksDBConsumeQueueTable#rangeQuery`, which builds N RocksDB keys for consecutive ConsumeQueue offsets and then calls `multiGet`.
This enhancement proposes optimizing the RocksDB ConsumeQueue sequential read path by using a bounded `RocksIterator` scan for consecutive offsets, while preserving the existing external ConsumeQueue iterator semantics.
### Motivation
Using RocksDB's native iterator scan for continuous offsets to improve performance.
### Describe the Solution You'd Like
Use a bounded RocksIterator scan in RocksDBConsumeQueueTable#rangeQuery to read continuous offsets while keeping existing ConsumeQueue iterator semantics unchanged.
### Describe Alternatives You've Considered
No
### Additional Context
_No response_
Contributor guide
Research direction
Start by reading RocksDBConsumeQueueTable#rangeQuery and tracing the iteratorWhenUseRocksdbConsumeQueue read path. Compare the existing multiGet behavior with a bounded RocksIterator scan for consecutive offsets, then verify that the external ConsumeQueue iterator semantics remain unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- database
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100