apache / apache/pulsar-client-go

[Bug] consumer under same namespace are blocked

Open
#1,362 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
745
Forks
389
Avg merge
3d 20h
Merged PRs (30d)
3

Description

#### Expected behavior

consumers under the same namespace are not blocked

#### Actual behavior

consumers under the same namespace are all blocked

#### Steps to reproduce

1.create several pulsar topics, they under same namespace, and all topic have consumers. send msg to these topics continuously.
2.create some consumers, this consumers contain **retry** and **dlq** queue, the code as below.
`

c, err := pulsarClient.Subscribe(pulsar.ConsumerOptions{
Topic: common.GetEventbridgeRuleTopicNameV2(target.Rule.Eventbridge.Name, target.Rule.Eventbridge.ID, target.Rule.Name, target.Rule.ID),
SubscriptionName: common.GetEventbridgeTargetSubscriptionNameV2(target.Name, target.ID),,
Type: pulsar.Shared,
RetryEnable: true,
NackBackoffPolicy: ExponentialNackBackoffRetryPolicy{},
DLQ: &pulsar.DLQPolicy{
MaxDeliveries: 20,
DeadLetterTopic: common.GetEventbridgeTargetDLQTopicNameV2(
common.GetEventbridgeRuleTopicNameV2(target.Rule.Eventbridge.Name, target.Rule.Eventbridge.ID, target.Rule.Name, target.Rule.ID),
target.Name,
target.ID,
),
RetryLetterTopic: common.GetEventbridgeTargetRetryTopicNameV2(
common.GetEventbridgeRuleTopicNameV2(target.Rule.Eventbridge.Name, target.Rule.Eventbridge.ID, target.Rule.Name, target.Rule.ID),
target.Name,
target.ID,
),
InitSubscriptionName: "default",
},
})

`

**common** package as below, very simple logic
`

func GetEventbridgeRuleTopicNameV2(eventbridgeName string, eventbridgeId int64, ruleName string, ruleId int64) string {
return fmt.Sprintf("persistent://eventbridge/%s-%d/rule-%s-%d", eventbridgeName, eventbridgeId, ruleName, ruleId)
}

func GetEventbridgeTargetSubscriptionNameV2(targetName string, targetId int64) string {
return fmt.Sprintf("target-%s-%d", targetName, targetId)
}

func GetEventbridgeTargetDLQTopicNameV2(topic string, targetName string, targetId int64) string {
return fmt.Sprintf("%s-target-%s-%d-DLQ", topic, targetName, targetId)
}

func GetEventbridgeRuleTopicNameV2(eventbridgeName string, eventbridgeId int64, ruleName string, ruleId int64) string {
return fmt.Sprintf("persistent://eventbridge/%s-%d/rule-%s-%d", eventbridgeName, eventbridgeId, ruleName, ruleId)
}

func GetEventbridgeTargetRetryTopicNameV2(topic string, targetName string, targetId int64) string {
return fmt.Sprintf("%s-target-%s-%d-RETRY", topic, targetName, targetId)
}

`
3. after consumer reviced msg, we assume handle msg occurs error, invoke **unAck()** method directly.
4. the producer sustain send message to topic, the message flow is very large, after several time, the consumer could not consumer any message, also under the same namespace, other topic's consumer could nerver consume any msg either(other topic msg flow is very small).

**we use pulsar-client-go 0.12.0, find above problem exist, but when we use 0.15.0, the problem is disappear
so I want to know why 0.12.0 have this problem, if there is some issue that linked this problem.**

the pulsar conf below, we set the unacked msg all are 0

![Image](https://github.com/user-attachments/assets/e26962e0-12ef-4e42-9c06-3563c77ce394)

#### System configuration
**Pulsar Server version**: both 2.8.1 and 3.0.6 have this problem
**Pulsar Go Client version**: 0.12.0

Contributor guide

Open the contributing guide

Research direction

Start with the Go client's Subscribe path and the interaction among RetryEnable, NackBackoffPolicy, DLQPolicy, and unAck in version 0.12.0. Compare that behavior with 0.15.0, focusing on consumers and retry or DLQ topics within one namespace; done means identifying the cause and adding a regression test showing that one consumer does not block unrelated consumers.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.