apache / apache/pulsar-client-go

add context control for pub cmd and add async method

Open
#1,147 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
745
Forks
389
Avg merge
3d 20h
Merged PRs (30d)
3

Description

**Is your feature request related to a problem? Please describe.**
add context control for pub cmd and add async method.
1. For go, there are many reasons for us to use context to control a request.
2. go has no keyword like `await`, but sometimes we may want it nonblock, eg: we pub a message without need to receive its result.
```
func (c *rpcClient) RequestToAnyBroker(requestID uint64, cmdType pb.BaseCommand_Type,
message proto.Message) (*RPCResult, error) {
var err error
var host *url.URL
var rpcResult *RPCResult
startTime := time.Now()
backoff := DefaultBackoff{100 * time.Millisecond}
// we can retry these requests because this kind of request is
// not specific to any particular broker
for time.Since(startTime) < c.requestTimeout {
host, err = c.serviceNameResolver.ResolveHost()
if err != nil {
c.log.WithError(err).Errorf("rpc client failed to resolve host")
return nil, err
}
rpcResult, err = c.Request(host, host, requestID, cmdType, message)
// success we got a response
if err == nil {
break
}

retryTime := backoff.Next()
c.log.Debugf("Retrying request in {%v} with timeout in {%v}", retryTime, c.requestTimeout)
time.Sleep(retryTime)
}

return rpcResult, err
}

func (c *rpcClient) Request(logicalAddr *url.URL, physicalAddr *url.URL, requestID uint64,
cmdType pb.BaseCommand_Type, message proto.Message) (*RPCResult, error) {
c.metrics.RPCRequestCount.Inc()
cnx, err := c.pool.GetConnection(logicalAddr, physicalAddr)
if err != nil {
return nil, err
}

ch := make(chan result, 1)

cnx.SendRequest(requestID, baseCommand(cmdType, message), func(response *pb.BaseCommand, err error) {
ch <- result{&RPCResult{
Cnx: cnx,
Response: response,
}, err}
})

timeoutCh := time.After(c.requestTimeout)
for {
select {
case res := <-ch:
// Ignoring producer not ready response.
// Continue to wait for the producer to create successfully
if res.error == nil && *res.RPCResult.Response.Type == pb.BaseCommand_PRODUCER_SUCCESS {
if !res.RPCResult.Response.ProducerSuccess.GetProducerReady() {
timeoutCh = nil
break
}
}
return res.RPCResult, res.error
case <-timeoutCh:
return nil, ErrRequestTimeOut
}
}
}
```

**Describe the solution you'd like**
the above code is confused. Retry should seperate from a single request. eg:
```
func (c *rpcClient) WithRetry(func()){
}
this would avoid the above code that inline Request method to RequestToAnyBroker :
for{
for{
select{
case <-ch1:
case <- ch2:
}
}
}
```

Contributor guide

Open the contributing guide

Research direction

Start by tracing rpcClient.RequestToAnyBroker and Request, including their retry, timeout, and callback behavior. Compare these entry points with the proposed WithRetry(func()) idea and define how context cancellation, asynchronous publication, and separated retry behavior should be represented; done requires an agreed API and corresponding verification, but no test file is named.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
api, backend-api-design, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.