apache / apache/pulsar

[Question][scalable-topics] How should a pooled client unregister a consumer?

Open
#26,272 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

## Context

A controller-managed scalable consumer has durable membership keyed by topic, subscription, and consumer name. The broker already has internal unregister operations that delete this membership and rebalance peers.

The binary protocol exposes subscribe, subscribe response, and assignment update, but no scalable-consumer close or unregister command. `CommandScalableTopicClose` closes only a DAG watch.

The Java V5 `ScalableConsumerClient.close()` removes the local callback from `ClientCnx` and sends no broker command. The broker starts its grace timer only when `ServerCnx.channelInactive()` observes the physical connection closing.

With two scalable consumers sharing one pooled `ClientCnx`, closing one consumer can therefore leave its broker membership active indefinitely because the other consumer keeps the connection non-idle. The closed member may continue reserving assigned segments.

There also appears to be a fencing risk if the same consumer name reattaches on a new connection while the old pooled connection remains alive: the old connection can later report a disconnect for that name.

## Questions

1. Is logical `StreamConsumer.close()` intended to mean permanent group departure, or only local detach until the pooled connection eventually closes?
2. Should PIP-468 add an explicit scalable-consumer unregister command and response?
3. Should explicit unregister bypass reconnect grace, remove durable membership, and rebalance peers immediately?
4. Which identity or generation should fence a stale old connection's later disconnect?
5. Would a test with two scalable consumers sharing one `ClientCnx`, closing one while preserving the other, represent the intended contract?

References: PIP-468, `ScalableConsumerClient.close`, `ServerCnx.channelInactive`, and `SubscriptionCoordinator.unregisterConsumer` on current `master`.

Contributor guide

Open the contributing guide

Research direction

Start with PIP-468 and the referenced ScalableConsumerClient.close, ServerCnx.channelInactive, and SubscriptionCoordinator.unregisterConsumer paths. Reproduce the proposed two-consumer shared-ClientCnx scenario, then document the agreed close, unregister, fencing, and reconnect contract; done includes a test representing that contract.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.