apache / apache/pulsar

[Broker & Bookie] Configure Bookkeeper client & server backpressure by default

Open
#10,439 8 comments 1 reaction 0 assignees View on GitHub
lifecycle/stale type/enhancement
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 22h
Merged PRs (30d)
142

Description

**Is your enhancement request related to a problem? Please describe.**

Pulsar doesn't take advantage of Bookkeeper's client-side and server-side backpressure features.
https://github.com/apache/bookkeeper/issues/1086 and https://github.com/apache/bookkeeper/issues/1409 are the original Bookkeeper issues about the lack of backpressure handling. Similar problems exist with Pulsar since backpressure handling isn't configured. One of the reported issues is #9562 .

**Describe the solution you'd like**

Configure `waitTimeoutOnBackpressureMs` in the Bookkeeper client configuration (Pulsar broker side) and `waitTimeoutOnResponseBackpressureMs`, `maxAddsInProgressLimit`, `maxReadsInProgressLimit` in the Bookkeeper server ("Bookie") configuration with sane default values.

**Additional context**

#9562

Contributor guide

Open the contributing guide

Research direction

Start by reviewing Pulsar issue #9562 and the referenced Bookkeeper issues #1086 and #1409 to understand the reported backpressure behavior. Trace the broker-side Bookkeeper client configuration and Bookie server configuration, then define and apply sane defaults for waitTimeoutOnBackpressureMs, waitTimeoutOnResponseBackpressureMs, maxAddsInProgressLimit, and maxReadsInProgressLimit; done means both sides use documented defaults and the related behavior is covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.