apache / apache/pulsar-client-go
Resend after reconnection cause reproduce of the same msg
- Dominant language
- Go
- Stars
- 745
- Forks
- 389
- Avg merge
- 3d 20h
- Merged PRs (30d)
- 3
Description
https://github.com/apache/pulsar-client-go/blob/965045aa0d2c077826f64d0b88a5d5e55d9d50c4/pulsar/producer_partition.go#L272
After the reconnection of the producer, the msg in the pendingQueue will be resend.
But not every msg in pendingQueue hasn't send to broker, the resend logic here seems to deal with the problem that producer meet a network error when producing a msg which cause the msg doesn't really send to broker.
While, the code here
https://github.com/apache/pulsar-client-go/blob/965045aa0d2c077826f64d0b88a5d5e55d9d50c4/pulsar/producer_partition.go#L793
will also trigger the reconnection logic again and again, as well as the resend logic, which cause the consumer to consume the same msg with difference msgId
Contributor guide
Research direction
Start by reading the pendingQueue resend logic in pulsar/producer_partition.go around lines 272 and 793, then trace how reconnection is triggered after a network error. Reproduce the producer reconnection scenario with a consumer and verify that messages already sent to the broker are not delivered again with different message IDs while genuinely pending messages still receive the required retry.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100