apache / apache/pulsar-client-cpp

[feature request] C++ and Python client support consumer memory limits.

Offen
#93 1 Kommentar 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
C++
Sterne
71
Forks
90
Ø Merge
2 Std. 33 Min.
Gemergte PRs (30 T.)
3

Beschreibung

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Motivation

[PIP-74](https://github.com/apache/pulsar/wiki/PIP-74%3A-Pulsar-client-memory-limits)

C++ client consumers have not implemented memory limits yet.

### Solution

There are several key points to achieving this feature.
1. Reconstructing the consumer structure.
- https://github.com/apache/pulsar/pull/16969
- Abstract `ConsumerBase`, put together some common logic.
2. Refactoring single-consumer and MultiTopicConsumer transports type.
- In the current implementation of C++, messages are sent to multi-topic consumers by blocking queue + listener, which is an old historical implementation. If you want to achieve memory limits, you need to do the same as the current java implementation, let `MuiltiTopicConsumer` take the initiative to control the message.

### Alternatives

Here's an imperfect draft: https://github.com/apache/pulsar/pull/16933

### Anything else?

_No response_

### Are you willing to submit a PR?

- [x] I'm willing to submit a PR!

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Start with PIP-74 and the ConsumerBase and MultiTopicConsumer changes described in PR 16969, then compare the imperfect draft in PR 16933. Done means defining and implementing consumer memory-limit control for the requested C++ and Python clients, including the MultiTopicConsumer path.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
cpp, python
Bereich
backend-api-design, distributed-systems
Issue-Typ
Feature
Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Aktivitätsstatus
Veraltet
Klarheit
Muss geklärt werden
Anfängerfreundlichkeit
25/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.