cockroachdb / cockroachdb/cockroach
gossip: forwarding (redirecting) connection requests does not consider locality-advertise-addr
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
Each node has a maximum number of incoming connection it would accept, [determined](https://github.com/cockroachdb/cockroach/blob/ce3f78b16d2224bac35baf95a263d5102afdca72/pkg/gossip/gossip.go#L676) based on the number of nodes in the cluster (with 3 being the minimum). Once this number is reached, incoming connection attempts are rejected, and the response includes an [alternative node suggestion ](https://github.com/cockroachdb/cockroach/blob/aa85c69776da942b4d539891725b780ba995401e/pkg/gossip/server.go#L262)(chosen at random), which is called forwarding.
The relevant log lines here are:
- ["refusing gossip from"](https://github.com/cockroachdb/cockroach/blob/ce3f78b16d2224bac35baf95a263d5102afdca72/pkg/gossip/server.go#L277) on the gossip server side.
- ["received forward from" ](https://github.com/cockroachdb/cockroach/blob/ce3f78b16d2224bac35baf95a263d5102afdca72/pkg/gossip/client.go#L268 ) on the gossip client side.
The forward information includes the node id and address. But it does not include the locality-advertise-addr, which determines how other nodes should connect to this node based on their locality, and possibly also not the fallback address (advertise-addr). This could result in unsuccessful connect attempts. We should consider sending more info, or the node descriptor rather than one address.
Jira issue: CRDB-33509
gz#19179
Contributor guide
Assessment
This issue has not been assessed yet.