apache / apache/rocketmq-exporter
Support queue-level consumer metrics (diff / latency / consumer offset)
- Dominant language
- Java
- Stars
- 324
- Forks
- 178
- PR merge metrics
- No merged PRs in 30d
Description
**Is your feature request related to a problem? Please describe.**
All consumer-side metrics are currently aggregated to broker granularity, so there is no way to tell *which queue* of a topic is lagging. When a single queue is stuck (a slow message, a stuck consumer instance, or an uneven rebalance), `rocketmq_group_diff` only shows that the group as a whole is behind — the skew between queues is invisible in Prometheus, and one has to fall back to the console (`Topic -> Consumer detail`) to see per-queue `brokerOffset` / `consumerOffset` / `diff`.
Interestingly, the label list for queue granularity already exists in the code but is never used:
https://github.com/apache/rocketmq-exporter/blob/master/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java#L479-L481
```java
private static final List GROUP_PULL_LATENCY_LABEL_NAMES = Arrays.asList(
"cluster", "broker", "topic", "group", "queueid"
);
```
**Describe the solution you'd like**
The per-queue data is already fetched and iterated in `MetricsCollectTask#collectConsumerOffset`: `consumeStats.getOffsetTable()` is a `Map`, and the per-queue `lagTime` is already computed one queue at a time. Both loops then fold the values into `HashMap` and the `queueId` is dropped.
So exposing queue granularity needs no new admin call at all — `queryMsgByOffset` is already invoked once per queue, and the proposal keeps that unchanged. Suggested metrics:
| metric | meaning |
| --- | --- |
| `rocketmq_queue_group_diff` | per-queue unconsumed messages (`brokerOffset - consumerOffset`) |
| `rocketmq_queue_group_get_latency_by_storetime` | per-queue consume latency (ms) |
| `rocketmq_queue_consumer_offset` | per-queue consumer offset |
**Cardinality concern and how it is addressed**
Queue-level series multiply the existing consumer series by the number of queues per broker, which is not acceptable to enable globally on a large cluster. The proposal therefore keeps it **off by default** and gated by an explicit topic whitelist:
```yaml
rocketmq:
config:
queueLevelTopics: "topic-a,topic-b" # empty = disabled (default), "*" = all topics
```
Existing broker-level metrics are untouched, so current dashboards and alerts keep working.
**Additional context**
I have a working implementation (pure addition, no existing line modified, with unit tests for the whitelist parsing) and will open a PR referencing this issue.
Contributor guide
Research direction
Start in RMQMetricsCollector.java around GROUP_PULL_LATENCY_LABEL_NAMES and trace MetricsCollectTask#collectConsumerOffset, including its offset-table and lag-time loops. Review the queueLevelTopics configuration and the mentioned whitelist unit tests. Done means queue-level metrics are exposed only for whitelisted topics, remain disabled by default, and existing broker-level metrics are unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, prometheus
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100