apache / apache/pulsar

[Feature Request] Expose FunctionCollectorRegistry through Context API for custom metrics

Open
#24,853 0 comments 0 reactions 0 assignees View on GitHub
type/enhancement
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before reporting

- [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Motivation

Currently, Pulsar Functions provide `context.recordMetric()` for custom metrics, but this requires enabling stateful functions with BookKeeper state storage. This adds significant infrastructure complexity and performance overhead for simple use cases like incrementing counters.

## Problem

Developers who want to expose custom Prometheus metrics without state storage overhead have no supported way to:
1. Register custom Prometheus collectors (Counter, Gauge, Histogram, etc.)
2. Access the internal FunctionCollectorRegistry that Pulsar uses

The Context API only exposes `recordMetric()` which requires state storage.

### Solution

## Proposed Solution

Add a method to the Context API to access the FunctionCollectorRegistry:

```java
public interface Context {
// Existing methods...

/**
* Get the CollectorRegistry for registering custom Prometheus metrics.
* This allows functions to register Counters, Gauges, Histograms, etc.
* without requiring state storage to be enabled.
*
* @return The function's CollectorRegistry
* @since 4.x.x
*/
CollectorRegistry getMetricsRegistry();
}
```

Use Cases

1. High-frequency counters (per-message metrics) without state storage overhead
2. Custom gauges, histograms, and summaries
3. Metrics with custom labels
4. Integration with existing Prometheus instrumentation libraries

Impact

- Low: Only adds one method to Context interface
- Backward compatible: Existing functions continue to work
- No breaking changes

### Alternatives

The obvious alternative is to use context.IncMetric() or similar. But this isn't suitable for the types of metrics I am interested in, e.g. a counter incremented for every message - it would involve network traffic, persistence and replication.

### Anything else?

_No response_

### Are you willing to submit a PR?

- [x] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start at the Context API and trace existing FunctionCollectorRegistry usage and Context implementations. Check related function metrics tests, then verify that custom Prometheus collectors can be registered without state storage while existing functions remain compatible.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, prometheus
Domain
api, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.