apache / apache/rocketmq

[Enhancement] Support suppressing minimal-value metrics to reduce export payload

Open
#11,078 3 comments 0 reactions 0 assignees View on GitHub
type/enhancement
Dominant language
Java
Stars
22.6k
Forks
12k
Avg merge
3d 1h
Merged PRs (30d)
27

Description

### Before Creating the Enhancement Request

- [x] I have confirmed that this should be classified as an enhancement rather than a bug/feature.

### Summary

This PR introduces an opt-in broker config `suppressMinValueMetrics` (default `false`, backward compatible) that suppresses exporting data points whose value is not greater than a
minimal threshold.

### Motivation

The consumer lag gauges are asynchronous ObservableGauges that emit one data point per consumer_group × topic × is_retry combination. In deployments with many consumer groups and topics, the vast majority of these combinations are idle at any given sampling instant, so their values are 0 — yet they are still recorded and exported on every collection cycle. This inflates both metric cardinality and the OTLP export payload.

In production, we observed zero-valued points accounting for ~95% of gauge data points and the bulk of the payload. Suppressing them is safe as long as the monitoring pipeline tolerates discontinuous series.

### Describe the Solution You'd Like

- `BrokerConfig`: add `suppressMinValueMetrics` (default `false`).
- `BrokerMetricsManager`: add a general guard `shouldRecordValue(currentValue, minValue)`.
When suppression is enabled, a data point is recorded only if `currentValue > minValue`.
- Apply the guard at the record site of the 5 consumer lag gauges, all passing `minValue = 0` for now, i.e. zero-valued points are dropped.

### Describe Alternatives You've Considered

.

### Additional Context

When enabled, threshold-based dashboards/alerts (e.g. `lag > N`) are unaffected; the only difference is
that idle series appear as gaps instead of a flat zero line. Therefore, before turning this on, make sure
there are **no alert rules based on `absent()` / `absent_over_time()`** (or any equivalent "no data" detection) targeting these metrics.

Contributor guide

Open the contributing guide

Research direction

Start by locating BrokerConfig and BrokerMetricsManager, then find the five consumer lag gauge record sites described in the issue. Confirm the new option defaults to false and that enabling it suppresses values at or below the stated minimum while preserving existing behavior when disabled; also check the documented absent-series alert caveat.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, observability-sre
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.