apache / apache/pulsar

[PIP-214][broker]Add broker level metrics statistics and expose to prometheus

Open
#18,056 4 comments 0 reactions 0 assignees View on GitHub
Stale
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Motivation

Currently, pulsar does not statistics broker level metrics, and all relevant metrics are 0 by default.

When the number of topic partitions reaches more than 100000 or even millions, and the topic level metrics are exposed (exposeTopicLevelMetricsInPrometheus=true), if you want to query the metrics of the broker dimension, you need to summarize all topics, and the performance becomes very poor, or even the results cannot be queried from the promethus. Common broker level metrics include:
```
pulsar_topics_count
pulsar_subscriptions_count
pulsar_producers_count
pulsar_consumers_count
pulsar_rate_in
pulsar_rate_out
pulsar_throughput_in
pulsar_throughput_out
pulsar_storage_size
pulsar_storage_logical_size
pulsar_storage_write_rate
pulsar_storage_read_rate
pulsar_msg_backlog
```

We need to statistics the metrics of the broker dimension and expose them to prometheus to improve the performance of the monitoring of the broker dimension. Modify the original metrics name as follows:
```
pulsar_broker_topics_count
pulsar_broker_subscriptions_count
pulsar_broker_producers_count
pulsar_broker_consumers_count
pulsar_broker_rate_in
pulsar_broker_rate_out
pulsar_broker_throughput_in
pulsar_broker_throughput_out
pulsar_broker_storage_size
pulsar_broker_storage_logical_size
pulsar_broker_storage_write_rate
pulsar_broker_storage_read_rate
pulsar_broker_msg_backlog
```

### Solution

_No response_

### Alternatives

_No response_

### Anything else?

_No response_

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!
PR:https://github.com/apache/pulsar/pull/18116

Contributor guide

Open the contributing guide

Research direction

The issue names no files or tests and links PR 18116, which should be reviewed first. Trace the existing broker-level Prometheus metrics path and compare it with the listed broker metric names; done means those broker statistics are exposed with the requested names and monitoring performance improves.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, prometheus
Domain
backend, observability-sre
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.