apache / apache/pulsar

[improve][ws] WebSocket proxy producers have no backpressure: the client memory limit is always disabled

Open
#26,373 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Motivation

Producers created by the WebSocket proxy have no backpressure at all out of the box, so a client that publishes faster than the broker acknowledges makes the shared proxy JVM buffer without limit.

Both of the mechanisms that would bound it are off:

- **Byte-based.** `WebSocketService.createClientInstance` passes `webSocketPulsarClientMemoryLimitInMB` straight to `ClientBuilder.memoryLimit`, and that config is an `int` defaulting to `0`, which the client reads as "no memory limit". #22666 added the setting, but because `0` is both its default and the client's "disabled" value, the client's own 64M default can never take effect — the proxy always overrides it with 0. Before #22666 the call was a hard-coded `.memoryLimit(0, SizeUnit.BYTES)`, so the behaviour is unchanged from then.
- **Message-count.** `ProducerHandler.getProducerBuilder` uses the no-argument `PulsarClient.newProducer()`, and `maxPendingMessages` is left at its default of `0`.

`ProducerImpl` only creates its pending-message semaphore `if (conf.getMaxPendingMessages() > 0)`, and `MemoryLimitController` never blocks when its limit is 0, so neither path bounds the queue. The proxy also sets `blockIfQueueFull(false)` specifically so its threads are not blocked when a lot of messages are pending — which is the right choice, but it only takes effect once something actually bounds the queue.

### Solution

Give the proxy byte-based backpressure by default, and let an operator opt out explicitly:

- Make `webSocketPulsarClientMemoryLimitInMB` an `Integer` with no default in `WebSocketProxyConfiguration` and `ServiceConfiguration`, and call `ClientBuilder.memoryLimit` only when it is set. Unset then means "use the client default" (64M) rather than "disabled", while `0` keeps meaning "disabled" for anyone who wants that. This is the same unset-vs-explicit-0 distinction as in #26342.
- Once the memory limit applies, review the `maxPendingMessages` query parameter: it is remote-supplied, so a value of `0` would ask for an unbounded pending queue. Whether the proxy should accept that from a client is worth deciding on its own.

Note this is a behaviour change for existing deployments that never set the config: their WebSocket producers become bounded by 64M of client memory, and a client that outruns the broker will start getting failed `ProducerAck` responses instead of the proxy buffering indefinitely. That is the intent, but it is worth calling out in the release notes.

Context: this came up in #26342, which fixes the equivalent problem on the client side (the pending-message defaults applied when a client's memory limit is disabled). A guard for the proxy's query parameter was included there at first and reverted, so the proxy is handled on its own here.

### Alternatives

Bounding the proxy's producers with `maxPendingMessages` instead. The memory limit is the better fit: the proxy multiplexes many producers over one client, and what matters is the total memory held in the JVM, not the queue depth of any single one.

### Anything else?

_No response_

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start with WebSocketService.createClientInstance, WebSocketProxyConfiguration, and ServiceConfiguration to trace how the memory setting reaches ClientBuilder.memoryLimit. Then inspect ProducerHandler.getProducerBuilder and the pending-message behavior; done means an unset setting uses the client default, explicit 0 disables it, and the proxy's remote pending-message option is addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.