apache / apache/pulsar

Support configuring the BatcherBuilder through a producer property

Open
#11,036 1 comment 0 reactions 0 assignees View on GitHub
lifecycle/stale type/enhancement
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 22h
Merged PRs (30d)
142

Description

**Is your enhancement request related to a problem? Please describe.**
Consumers support `KEY_SHARED` subscriptions. For this to work with producer batching, the producer needs to use a `KEY_BASED` `BatcherBuilder`. The `subscriptionType` of the consumer is currently configurable through properties. The `BatcherBuilder` used by the producer however, is only configurable through code. It would be nice if this could also be done by property.

**Describe the solution you'd like**
Have a configuration property (e.g. `batcherBuilderType`) which supports values `DEFAULT` and `KEY_BASED` for configuring the `BatcherBuilder` used by the producer.

**Describe alternatives you've considered**
None

**Additional context**
Third party products that use pulsar (e.g. debezium) expose the pulsar config through properties. Only being able to configure this in-code is inconvenient in such cases.

Contributor guide

Open the contributing guide

Research direction

No files or tests are named. Start by tracing how producer properties and the producer's BatcherBuilder are currently configured, alongside the existing consumer subscriptionType property handling. Done means a producer property accepts DEFAULT and KEY_BASED and selects the corresponding BatcherBuilder, with coverage for both values.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.