RPC connection pool should be aware of application-level errors
Open
theme/reliability
- Dominant language
- Go
- Stars
- 30.1k
- Forks
- 4.6k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 43
Description
Currently the RPC connection pool (in `agent/pool/` and some surrounding code) is only aware of connection errors.
If a server is accepting connections, but all RPC requests are failing, the connection pool will not be aware of these failures.
We would also like to introduce a new error that servers will send back when they are not yet ready to handle RPC requests (possibly because they are in the process of loading a snapshot).
https://github.com/hashicorp/consul/blob/v1.10.1/agent/pool/pool.go#L591-L621 will need to look at `reply` and if its an error decide if it should `p.clearConn` or not.
This may be related to #10739
Contributor guide
Assessment
This issue has not been assessed yet.