server: test that locality addr is always used

**Is your feature request related to a problem? Please describe.**
With locality addresses, the expectation is to always use the locality address when communicating with a node rather than the advertised-addr if the locality address is set. In a recent bug #117528 we noticed that the code in the UI was not using this correctly and it appeared there is another place that is using this incorrectly: https://github.com/andrewbaptist/cockroach/blob/3da2bec0288fe058207d146b391674d243f9889d/pkg/server/tenant.go#L1174

We should create an automated test that covers these cases.

**Describe the solution you'd like**
A solution like this could work:
Create a 6 node cluster in roachprod and start all the nodes with the default settings
```
roachprod create -n 6 $CLUSTER
```
Start each node with the locality set to the nodes normal address, but the advertise address set to a non-routable address:
e.g.
```
roachprod start $CLUSTER:n --args "--advertise-addr=192.168.1.1" --args "--locality-advertise-addr=cloud=gce@10.142.0.101"
```

**Describe alternatives you've considered**
An alternative to test this is to actually create different localities within roachprod and make them non-routable, but this is more difficult.

**Additional context**
We know of at least a couple places in our codebase that are still wrong with regards to this, finding and fixing these is important.

Jira issue: CRDB-35304