apache / apache/pulsar-client-go

Resend after reconnection cause reproduce of the same msg

Open
#731 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
745
Forks
389
Avg merge
3d 20h
Merged PRs (30d)
3

Description

https://github.com/apache/pulsar-client-go/blob/965045aa0d2c077826f64d0b88a5d5e55d9d50c4/pulsar/producer_partition.go#L272

After the reconnection of the producer, the msg in the pendingQueue will be resend.
But not every msg in pendingQueue hasn't send to broker, the resend logic here seems to deal with the problem that producer meet a network error when producing a msg which cause the msg doesn't really send to broker.

While, the code here

https://github.com/apache/pulsar-client-go/blob/965045aa0d2c077826f64d0b88a5d5e55d9d50c4/pulsar/producer_partition.go#L793

will also trigger the reconnection logic again and again, as well as the resend logic, which cause the consumer to consume the same msg with difference msgId

Contributor guide

Open the contributing guide

Research direction

Start by reading the pendingQueue resend logic in pulsar/producer_partition.go around lines 272 and 793, then trace how reconnection is triggered after a network error. Reproduce the producer reconnection scenario with a consumer and verify that messages already sent to the broker are not delivered again with different message IDs while genuinely pending messages still receive the required retry.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.