apache / apache/pulsar

PIP-235: Add metric for subscription backlog size

Open
#19,112 3 comments 0 reactions 0 assignees View on GitHub
Stale type/PIP
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Motivation

Now we have `pulsar_storage_backlog_size` for topic backlog size, user can create an alarm rule like `pulsar_storage_backlog_size > THRESHOLD`, typically this alarm is going to notify corresponding subscription owner, but it need extra process to identify subscriptions that backlog size exceed the threshold. So we could add a new metric for subscription back size.

#9302 Added subscription backlog size info for topic stats and a configuration `exposeSubscriptionBacklogSizeInPrometheus` (default false) for topic metrics, but the latter was unimplemented.

### Goal

New metric `pulsar_subscription_back_log_size`

### API Changes

Since `0` is a valid value for subscription backlog size, `change subscriptionStats.backlogSize to -1 from 0` if request the stats of topic with `subscriptionBacklogSize=false`.

### Implementation

If and only if `exposeSubscriptionBacklogSizeInPrometheus` is enabled, expose the metric as `pulsar_subscription_back_log_size`

### Alternatives

_No response_

### Anything else?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reviewing #9302 and the topic-stats path that introduced subscription backlog size, then trace the exposeSubscriptionBacklogSizeInPrometheus configuration into Prometheus metric generation. Done means the pulsar_subscription_back_log_size metric appears only when enabled, and subscriptionStats.backlogSize is -1 rather than 0 when subscription backlog stats are not requested.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, prometheus
Domain
distributed-systems, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.