cockroachdb / cockroachdb/cockroach

gossip: forwarding (redirecting) connection requests does not consider locality-advertise-addr

Open
#114,430 0 comments 0 reactions 0 assignees View on GitHub
branch-master branch-release-22.2 branch-release-23.1 branch-release-23.2 C-bug O-support P-3 T-kv
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Each node has a maximum number of incoming connection it would accept, [determined](https://github.com/cockroachdb/cockroach/blob/ce3f78b16d2224bac35baf95a263d5102afdca72/pkg/gossip/gossip.go#L676) based on the number of nodes in the cluster (with 3 being the minimum). Once this number is reached, incoming connection attempts are rejected, and the response includes an [alternative node suggestion ](https://github.com/cockroachdb/cockroach/blob/aa85c69776da942b4d539891725b780ba995401e/pkg/gossip/server.go#L262)(chosen at random), which is called forwarding.

The relevant log lines here are:
- ["refusing gossip from"](https://github.com/cockroachdb/cockroach/blob/ce3f78b16d2224bac35baf95a263d5102afdca72/pkg/gossip/server.go#L277) on the gossip server side.
- ["received forward from" ](https://github.com/cockroachdb/cockroach/blob/ce3f78b16d2224bac35baf95a263d5102afdca72/pkg/gossip/client.go#L268 ) on the gossip client side.

The forward information includes the node id and address. But it does not include the locality-advertise-addr, which determines how other nodes should connect to this node based on their locality, and possibly also not the fallback address (advertise-addr). This could result in unsuccessful connect attempts. We should consider sending more info, or the node descriptor rather than one address.

Jira issue: CRDB-33509

gz#19179

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.