DestinationPeer not working for exported service across peered cluster.
- Dominant language
- Go
- Stars
- 30.1k
- Forks
- 4.6k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 43
Description
#### Overview of the Issue
Exported services across peered cluster are unable to communicate. Listeners do not accept , `destination_peer` parameter in upstream job.
---
#### Reproduction Steps
1) Nomad based consul clusters, peered.
2) Mesh and mesh-gateways in each of the clusters with ACL enabled and tokens with policies as shown below
```
service_prefix "mesh-gateway" { # consul service is registered under the name mesh-gateway.
policy = "write"
}
service_prefix "" {
policy = "write"
}
agent_prefix "" {
policy = "read"
}
node_prefix "" {
policy = "read"
}
mesh = "write"
peering = "read"
```
3) Setup service-intentions, exported-service, proxy-default objects as described [here](https://developer.hashicorp.com/consul/docs/connect/config-entries/exported-services#reading-services) and [here](https://developer.hashicorp.com/consul/docs/connect/cluster-peering/usage/establish-cluster-peering).
4) 2 hello-world service setup as nomad jobs with consul connect along with upstream setup (I think I am facing issue here).
5) ACL enabled tokens for services to communicate with each other. Attaching policy below
```
service_prefix "" {
policy = "read"
}
node_prefix "" {
policy = "read"
}
```
### Consul info for both Client and Server
```
agent:
check_monitors = 0
check_ttls = 0
checks = 0
services = 0
build:
prerelease =
revision = bd257019
version = 1.14.3
version_metadata =
consul:
acl = enabled
bootstrap = false
known_datacenters = 1
leader = true
server = true
runtime:
arch = arm64
cpu_count = 2
goroutines = 310
max_procs = 2
os = linux
version = go1.19.4
```
### Operating system and Environment details
OS: Ubuntu ,
Architecture: arm64
### Observations:
1) Mesh gateways are deployed successfully.
2) Peering via UI is successful. and I can see the exported/imported services in the UI.
3) Health check apis for exported services pass successfully.
4) consul catalog services , however doesn't show the exported services. I am not sure if it should?
I am unable to debug is why the upstream section in my exported services is unable to accept `destination_peer` parameter. The nomad job simply rejects it while applying. This is how my the sidecar initializes the listener.
`
[2023 22:50:40.906][1][info][upstream] [source/server/lds_api.cc:82] lds: add/update listener demo-app-cluster-2?dc=cluster-2:127.0.0.1:9090 `
What the connect section looks like:
```
"Connect": {
"Native": false,
"SidecarService": {
"Tags": null,
"Port": "",
"Proxy": {
"LocalServiceAddress": "",
"LocalServicePort": 8081,
"Upstreams": [
{
"DestinationName": "demo-app-cluster-2",
"DestinationNamespace": "",
"DestinationPeer":"cluster-2"
"LocalBindPort": 9090,
"Datacenter": "cluster-1",
"LocalBindAddress": "127.0.0.1",
"MeshGateway": {
"Mode": "local"
}
}
],
"Expose": null,
"Config": null
},
},
"SidecarTask": {...},
"Gateway": null
},
```
Contributor guide
Assessment
This issue has not been assessed yet.