apache / apache/pulsar

produce message supports rate limit in fail-fast

Open
#15,038 7 comments 0 reactions 0 assignees View on GitHub
lifecycle/stale Stale type/feature
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

**Is your feature request related to a problem? Please describe.**

now if one topic was rate limited, we will use disableCnxAutoRead, which will not read data from buffer.
one topic sends a message in t1, we disabled then, and in t2, we start to read, and processed (maybe t2-t1 > the timeout of users)

but when we processed in t3, when we send result to producer, it is timeout.

**Describe the solution you'd like**

we can choose the rate limit stratery, failfast or disableautoread

**Describe alternatives you've considered**

implement the fail fast mode, when rate limited, we will send servererror to producer

**Additional context**
Add any other context or screenshots about the feature request here.

when we use disableCnxAutoRead , sometimes ,if producers close the connection by close producer, but we cannot read the request, they we cannot close the connection, there will be many close-wait in broker

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points. Start by tracing broker rate-limit handling, disableCnxAutoRead, and the producer response path; determine where a fail-fast strategy can be selected and how a rate-limited request should return a server error. Done means the selected strategy avoids delayed producer timeouts and connections can close cleanly.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.