[Enhancement] Selective double-write in CombineConsumeQueueStore
- Dominant language
- Java
- Stars
- 22.6k
- Forks
- 12k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 26
Description
### Before Creating the Enhancement Request
- [x] I have confirmed that this should be classified as an enhancement rather than a bug/feature.
### Summary
CombineConsumeQueueStore was originally designed as a transitional state in which every topic is double-written to both the file-based ConsumeQueue and the RocksDB ConsumeQueue. As it has gradually become a long-running state in practice, we observed that not all topics actually need double-write. In particular, LITE topics need it more than ordinary topics, because LMQ is required to run on top of the RocksDB ConsumeQueue.
This is the original motivation for introducing selective double-write.
To avoid an incompatible breaking change, we introduce a new opt-in switch `rocksdbCQSelectiveDoubleWriteEnable` as an additional control rather than altering the existing default behavior. Existing deployments are therefore unaffected unless they explicitly opt in.
### Motivation
.
### Describe the Solution You'd Like
.
### Describe Alternatives You've Considered
.
### Additional Context
.
Contributor guide
Research direction
Start by locating CombineConsumeQueueStore and the rocksdbCQSelectiveDoubleWriteEnable switch, then trace how topics are written to the file-based and RocksDB ConsumeQueues. Clarify the intended opt-in behavior for LITE and ordinary topics while preserving the existing default, and define tests that demonstrate both modes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100