apache / apache/rocketmq

Make RocksDB ConsumeQueue size amplification configurable

Open
#10,344 1 comment 0 reactions 0 assignees View on GitHub
type/enhancement
Dominant language
Java
Stars
22.6k
Forks
12k
Avg merge
2d 20h
Merged PRs (30d)
26

Description

### Before Creating the Enhancement Request

- [x] I have confirmed that this should be classified as an enhancement rather than a bug/feature.

### Summary

Make RocksDB ConsumeQueue universal compaction size amplification configurable. This gives operators a CQ only tuning option while keeping the current default behavior.

### Motivation

RocksDB ConsumeQueue currently uses a fixed universal compaction max size amplification percent of 25. In environments with high compaction pressure operators may need to allow more temporary space amplification in exchange for fewer compactions and lower CPU pressure. A broker side config makes this tunable without changing the default.

### Describe the Solution You'd Like

Add a MessageStoreConfig option rocksdbMaxSizeAmplificationPercent with default value 25. Use it only when building the RocksDB ConsumeQueue column family universal compaction options. Do not change POP Index Trans Timer or RocksDB DBOptions defaults.

### Describe Alternatives You've Considered

Keep the value hard coded which requires a code change for deployments that need a different compaction tradeoff. Broader RocksDB option parameterization was also considered but it is not needed for this scoped CQ improvement.

### Additional Context

Scope is intentionally limited to RocksDB ConsumeQueue. Related PR will follow after this issue is created.

Contributor guide

Open the contributing guide

Research direction

Start with MessageStoreConfig to locate the new rocksdbMaxSizeAmplificationPercent setting and then trace how RocksDB ConsumeQueue builds its column-family universal compaction options. Keep the default at 25 and apply the setting only to that ConsumeQueue configuration, leaving POP Index Trans Timer and DBOptions defaults unchanged; verify with the relevant existing tests or build checks.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.