apache / apache/pulsar

[imporve] Supporting streaming metrics collection based on OpenTelemetry

Open
#24,480 0 comments 0 reactions 0 assignees View on GitHub
type/enhancement
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before reporting

- [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Motivation

In scenarios with massive numbers of Topics, enabling Topic-level Metrics and using Prometheus for collection can cause significant memory fluctuations on the Broker, frequently triggering GC. We can implement streaming Metrics collection based on OpenTelemetry and reuse objects to solve this problem. If someone thinks this feature is worthwhile, I can open a PIP.

### Solution

_No response_

### Alternatives

_No response_

### Anything else?

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are named. Start by locating the Broker's Topic-level Metrics implementation and its Prometheus collection path, then investigate how OpenTelemetry streaming collection and object reuse could fit. The issue does not define acceptance criteria; completion would require an agreed design or PIP before implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, prometheus
Domain
observability-sre
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.