apache / apache/pulsar-client-go

retry producer creation upon error after successful topic lookup

Open
#1,138 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
745
Forks
389
Avg merge
3d 20h
Merged PRs (30d)
3

Description

#### Expected behavior

In the newPartitionProducer() function, there should be a retry of grabCnx(). It will be similar to the reconnectToBroker's grabCnx() retry logic.

Java producer has this retry logic.

#### Actual behavior

At the producer creation call, after a successful topic lookup at grabCnx() in producer_partition.go, if there is a network issue before the COMMAND to create producer sent, the grabCnx() will exit without retry.

We had frequent failures upon the initial producer creation.

#### Steps to reproduce

It's tricky to reproduce. But we observe the problem more frequently on Azure pod's initialization stage. After implementing the grabCnx() retry in the newPartitionProducer(), the problem has gone away. (Will do a PR)

#### System configuration
**Pulsar version**: 2.10

Contributor guide

Open the contributing guide

Research direction

Start in producer_partition.go at newPartitionProducer() and compare its grabCnx() handling with the retry logic in reconnectToBroker. Review the corresponding Java producer behavior, then verify that an initial producer creation retries when the connection fails after topic lookup but before the create-producer command is sent.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.