apache / apache/rocketmq-exporter

Support queue-level consumer metrics (diff / latency / consumer offset)

Open
#190 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
324
Forks
178
PR merge metrics
No merged PRs in 30d

Description

**Is your feature request related to a problem? Please describe.**

All consumer-side metrics are currently aggregated to broker granularity, so there is no way to tell *which queue* of a topic is lagging. When a single queue is stuck (a slow message, a stuck consumer instance, or an uneven rebalance), `rocketmq_group_diff` only shows that the group as a whole is behind — the skew between queues is invisible in Prometheus, and one has to fall back to the console (`Topic -> Consumer detail`) to see per-queue `brokerOffset` / `consumerOffset` / `diff`.

Interestingly, the label list for queue granularity already exists in the code but is never used:

https://github.com/apache/rocketmq-exporter/blob/master/src/main/java/org/apache/rocketmq/exporter/collector/RMQMetricsCollector.java#L479-L481

```java
private static final List GROUP_PULL_LATENCY_LABEL_NAMES = Arrays.asList(
"cluster", "broker", "topic", "group", "queueid"
);
```

**Describe the solution you'd like**

The per-queue data is already fetched and iterated in `MetricsCollectTask#collectConsumerOffset`: `consumeStats.getOffsetTable()` is a `Map`, and the per-queue `lagTime` is already computed one queue at a time. Both loops then fold the values into `HashMap` and the `queueId` is dropped.

So exposing queue granularity needs no new admin call at all — `queryMsgByOffset` is already invoked once per queue, and the proposal keeps that unchanged. Suggested metrics:

| metric | meaning |
| --- | --- |
| `rocketmq_queue_group_diff` | per-queue unconsumed messages (`brokerOffset - consumerOffset`) |
| `rocketmq_queue_group_get_latency_by_storetime` | per-queue consume latency (ms) |
| `rocketmq_queue_consumer_offset` | per-queue consumer offset |

**Cardinality concern and how it is addressed**

Queue-level series multiply the existing consumer series by the number of queues per broker, which is not acceptable to enable globally on a large cluster. The proposal therefore keeps it **off by default** and gated by an explicit topic whitelist:

```yaml
rocketmq:
config:
queueLevelTopics: "topic-a,topic-b" # empty = disabled (default), "*" = all topics
```

Existing broker-level metrics are untouched, so current dashboards and alerts keep working.

**Additional context**

I have a working implementation (pure addition, no existing line modified, with unit tests for the whitelist parsing) and will open a PR referencing this issue.

Contributor guide

Open the contributing guide

Research direction

Start in RMQMetricsCollector.java around GROUP_PULL_LATENCY_LABEL_NAMES and trace MetricsCollectTask#collectConsumerOffset, including its offset-table and lag-time loops. Review the queueLevelTopics configuration and the mentioned whitelist unit tests. Done means queue-level metrics are exposed only for whitelisted topics, remain disabled by default, and existing broker-level metrics are unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, prometheus
Domain
observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.