[Feature Request] Expose FunctionCollectorRegistry through Context API for custom metrics
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
### Search before reporting
- [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Motivation
Currently, Pulsar Functions provide `context.recordMetric()` for custom metrics, but this requires enabling stateful functions with BookKeeper state storage. This adds significant infrastructure complexity and performance overhead for simple use cases like incrementing counters.
## Problem
Developers who want to expose custom Prometheus metrics without state storage overhead have no supported way to:
1. Register custom Prometheus collectors (Counter, Gauge, Histogram, etc.)
2. Access the internal FunctionCollectorRegistry that Pulsar uses
The Context API only exposes `recordMetric()` which requires state storage.
### Solution
## Proposed Solution
Add a method to the Context API to access the FunctionCollectorRegistry:
```java
public interface Context {
// Existing methods...
/**
* Get the CollectorRegistry for registering custom Prometheus metrics.
* This allows functions to register Counters, Gauges, Histograms, etc.
* without requiring state storage to be enabled.
*
* @return The function's CollectorRegistry
* @since 4.x.x
*/
CollectorRegistry getMetricsRegistry();
}
```
Use Cases
1. High-frequency counters (per-message metrics) without state storage overhead
2. Custom gauges, histograms, and summaries
3. Metrics with custom labels
4. Integration with existing Prometheus instrumentation libraries
Impact
- Low: Only adds one method to Context interface
- Backward compatible: Existing functions continue to work
- No breaking changes
### Alternatives
The obvious alternative is to use context.IncMetric() or similar. But this isn't suitable for the types of metrics I am interested in, e.g. a counter incremented for every message - it would involve network traffic, persistence and replication.
### Anything else?
_No response_
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
Research direction
Start at the Context API and trace existing FunctionCollectorRegistry usage and Context implementations. Check related function metrics tests, then verify that custom Prometheus collectors can be registered without state storage while existing functions remain compatible.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, prometheus
- Domain
- api, observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100