[imporve] Supporting streaming metrics collection based on OpenTelemetry
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
### Search before reporting
- [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Motivation
In scenarios with massive numbers of Topics, enabling Topic-level Metrics and using Prometheus for collection can cause significant memory fluctuations on the Broker, frequently triggering GC. We can implement streaming Metrics collection based on OpenTelemetry and reuse objects to solve this problem. If someone thinks this feature is worthwhile, I can open a PIP.
### Solution
_No response_
### Alternatives
_No response_
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
Research direction
No files, tests, or entry points are named. Start by locating the Broker's Topic-level Metrics implementation and its Prometheus collection path, then investigate how OpenTelemetry streaming collection and object reuse could fit. The issue does not define acceptance criteria; completion would require an agreed design or PIP before implementation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, prometheus
- Domain
- observability-sre
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100