Discovery can't be reliably updated if requests are infrequent
- Dominant language
- Rust
- Stars
- 12.5k
- Forks
- 1.3k
- Avg merge
- 4d 7h
- Merged PRs (30d)
- 24
Description
## Bug Report
### Version
tonic-0.10.0
### Platform
Darwin sams-mbp.buffalo-yo.ts.net 23.1.0 Darwin Kernel Version 23.1.0: Mon Oct 9 21:27:24 PDT 2023; root:xnu-10002.41.9~6/RELEASE_ARM64_T6000 arm64
### Description
If you make infrequent requests to a service relative to how often the endpoints are updated, eventually Tonic's Channel fills up and the client ends up using expired Endpoints. This happens because the Channel's receiver is only polled after a request is made to the service.
I may be reading the layers of services/buffers here incorrectly, but here's my understanding of the root cause:
Tower's `Buffer` does not [poll its service](https://github.com/tower-rs/tower/blob/39adf5c509a1b2141f679654d8317524ca96b58b/tower/src/buffer/worker.rs#L168) until after it [receives a request](https://github.com/tower-rs/tower/blob/39adf5c509a1b2141f679654d8317524ca96b58b/tower/src/buffer/worker.rs#L154). The request originates from a [call to the service](https://github.com/tower-rs/tower/blob/39adf5c509a1b2141f679654d8317524ca96b58b/tower/src/buffer/service.rs#L122), not a message to the channel. That means that if requests are infrequently sent to the service, the channel will fill up and we'll be unable to update `Endpoint`s.
The API, especially with the provided executor, looks like the channel's receiver will be polled continuously, but instead the executor is used as an implementation detail in Tower's buffer. I'd have expected the Channel to be polled and the cache to be updated continuously even if no requests are made.
Here's a minimal repro:
```
#[tokio::test]
async fn test_something() -> anyhow::Result<()> {
let (channel, sender) = Channel::balance_channel::<
&'static str,
>(1);
let endpoint = Endpoint::from_static("https://localhost.invalid");
sender
.send(tower::discover::Change::Insert("service", endpoint.clone()))
.await?;
// This blocks forever because the channel is full
sender
.send(tower::discover::Change::Insert("service", endpoint.clone()))
.await?;
Ok(())
}
```
Contributor guide
Assessment
This issue has not been assessed yet.