apache / apache/pulsar

PIP-237: Make PulsarAdmin accessible in SinkContext and SourceContext

Open
#19,123 4 comments 0 reactions 0 assignees View on GitHub
Stale type/PIP
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Motivation

Currently, a Pulsar function can retrieve a `PulsarAdmin` instance from its `Context`. Unfortunately, Pulsar IO connectors cannot retrieve a `PulsarAdmin` instance from its `SinkContext` or `SourceContext` respectively.
It would be useful to get `PulsarAdmin` access in connectors without having to add additional connector configuration properties like the broker endpoint and manually instantiating `PulsarAdmin` from these.
Scenarios where a connector might need `PulsarAdmin`, would be e.g. managing schemas, or validating a compatible broker configuration at runtime.

### Goal

Enable IO connectors to access `PulsarAdmin` through their context.
Given the reasons above, I would like to either:
a) add a `getPulsarAdmin` method to the `SinkContext` and `SourceContext` interfaces.

b) move the `getPulsarAdmin` method that is part of the functions `Context` into the `BaseContext` class as both `Context` as well as `SinkContext` and `SourceContext` inherit from this class.

My personal preference would be to go with option b).

### API Changes

We are going to move the `getPulsarAdmin` method that is part of the functions `Context` into the `BaseContext` class as both `Context` as well as `SinkContext` and `SourceContext` inherit from this class.

### Implementation

`SinkContext` and `SourceContext` interfaces will be extended with a `getPulsarAdmin` method. The implementation can be provided either on their implementation class directly, or by moving the `Context#getPulsarAdmin` method into the `BaseContext`.

### Alternatives

An alternative to the proposed API changes above would be to add a `getPulsarAdmin` method to the `SinkContext` and `SourceContext` interfaces directly with their respective implementations.

### Anything else?

_No response_

Contributor guide

Open the contributing guide

Research direction

Review the Context, BaseContext, SinkContext, and SourceContext APIs first, then trace their implementations. Confirm how connector contexts currently expose services and whether the proposed shared PulsarAdmin access is compatible with the existing API structure; the work is done when both sink and source connectors can access PulsarAdmin through their contexts without extra endpoint configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.