apache / apache/rocketmq

[Enhancement] Enable "useSeparateRetryQueue" for priority topics

Open
#10,350 0 comments 0 reactions 0 assignees View on GitHub
type/enhancement
Dominant language
Java
Stars
22.6k
Forks
12k
Avg merge
3d 1h
Merged PRs (30d)
27

Description

### Before Creating the Enhancement Request

- [x] I have confirmed that this should be classified as an enhancement rather than a bug/feature.

### Summary

Add a new global switch to enable `useSeparateRetryQueue` only for priority topics. This prevents high-priority retry messages from being blocked by accumulated low-priority retry messages, while keeping normal topics unchanged.

新增一个全局开关,只为优先级 Topic 启用`useSeparateRetryQueue`。这样可以避免高优先级重试消息被大量低优先级重试消息阻塞,同时不影响普通 Topic 的现有行为。

### Motivation

RIP-80 introduces priority messages for POP consumption. Priority messages are routed to different queues by priority, so consumers can process high-priority messages before low-priority messages.

RIP-80 为 POP 消费引入了优先级消息。消息会按优先级路由到不同队列,消费者可以优先消费高优先级消息。

However, once messages enter the retry topic, this priority can be lost. For example, a retry queue may contain:

但是,当消息进入 retry topic 后,优先级可能会失效。例如,retry queue 中可能出现:

```text
| low | low | low | low | low | high |
```

The high-priority retry message has to wait for all earlier low-priority retry messages. This causes priority inversion in the retry topic.

高优先级重试消息必须等待前面的低优先级重试消息消费完成,导致 retry topic 中出现优先级反转。

### Describe the Solution You'd Like

Add a new global switch that enables `useSeparateRetryQueue` only for priority topics.

新增一个全局开关,只为优先级 Topic 启用`useSeparateRetryQueue`。

Possible config name:

可能的配置名:

```text
useSeparateRetryQueueForPriorityTopic=true
```

Expected rules:

预期规则:

1. If broker-level `useSeparateRetryQueue=true`, keep the current behavior: all topics use separate retry queues.
2. If `useSeparateRetryQueue=false` and `useSeparateRetryQueueForPriorityTopic=true`, only priority topics use separate retry queues.
3. Normal topics are unchanged unless `useSeparateRetryQueue` is enabled.


1. 如果 Broker 级 `useSeparateRetryQueue=true`,保持现有行为:所有 Topic 使用独立 retry queue。
2. 如果 `useSeparateRetryQueue=false` 且 `useSeparateRetryQueueForPriorityTopic=true`,只有优先级 Topic 使用独立 retry queue。
3. 普通 Topic 行为保持不变,除非开启 `useSeparateRetryQueue`。

### Describe Alternatives You've Considered

Deploy priority topics in a separate RocketMQ cluster and enable `useSeparateRetryQueue` for that cluster.

把优先级 Topic 拆分到独立的 RocketMQ 集群中,并在该集群开启 `useSeparateRetryQueue`

### Additional Context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by tracing the existing broker-level useSeparateRetryQueue handling and the priority-topic retry routing introduced by RIP-80. Define the new global switch so the three stated combinations preserve current behavior for normal topics and give priority topics separate retry queues when requested; add or update coverage for those combinations.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.