grpc / grpc/grpc-rust

Client with a `balance_channel` keeps returning errors after servers have started.

Open
#1,147 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
12.5k
Forks
1.3k
Avg merge
4d 7h
Merged PRs (30d)
24

Description

## Bug Report
### Version

```
tonic-balance-channel-with-lazy-connection v0.1.0 (/Users/francois.massot@contentsquare-ext.com/Documents/quickwit/repos/tonic-balance-channel-with-lazy-connection)
└── tonic v0.8.2 (https://github.com/hyperium/tonic#33e22bbc)
└── tonic-build v0.8.2 (https://github.com/hyperium/tonic#33e22bbc)
```

### Platform

`Darwin MacBook-Pro-de-Francois.local 21.6.0 Darwin Kernel Version 21.6.0: Mon Aug 22 20:19:52 PDT 2022; root:xnu-8020.140.49~2/RELEASE_ARM64_T6000 arm64`

### Crates

`tonic`

### Description

When using a client with a `balance_channel` with `N` endpoints, I observed the following:
- I send a request to the server that has not yet started. I'm receiving an error which is fine.
- I start the server.
- I send a request again, then I got an error. I need to send typically between 2 and 6 requests to get an Ok result finally.

I wrote a test here: https://github.com/fmassot/tonic-balance-channel-with-lazy-connection/blob/main/src/main.rs#L25

When digging into the code, the issue comes from the way errors are handled with lazy connections: https://github.com/hyperium/tonic/blob/master/tonic/src/transport/service/reconnect.rs#L63-L155

Note that if there is one endpoint, there is no problem. This comes from the fact that, when servers are not started, the `poll_ready` will be executed on all `Reconnect`... and we will end up with `N` `Reconnect` instances with errors. Only one will answer and return the error. The other `N-1` will keep the error and return it on the next call.

### Solution

I can push a PR to fix that but I want to have your feedback before.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.