ingress-gateway: non-deterministic selection of cluster config with same service name
- Dominant language
- Go
- Stars
- 30.1k
- Forks
- 4.6k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 43
Description
#### Overview of the Issue
In an ingress-gateway config entry, when two listeners, one of them is wildcard, have different configurations, e.g., host, requestheader, consul creates the upstream cluster of envoy configuration by randomly pick one from the listeners.
This is most likely due to the map structure when creating the proxycfg:
https://github.com/hashicorp/consul/blob/d9fe3578acdc19b7024ff4927e1e93a43b2d78ed/agent/proxycfg/ingress_gateway.go#L125
#### Reproduction Steps
create an ingress-gateway entry with one wildcard listener:
```hcl
Listeners = [
{
Port = 8080
Protocol = "http"
Services = [
{
Name = "*"
MaxConnections = 2048
}
]
},
{
Port = 8081
Protocol = "http"
Services = [
{
Name = "foo"
MaxConnections = 4096
}
]
},
```
Run many times, and the `max_connections` of the upstream cluster in envoy config will change between 4096 and 2048.
**Update**
After some further investigation, the problem seems to be how we store the gateway services in the state store. In the above example, the service `foo` will be added to the `tableGatewayServices` twice, indexed by service name and listener port: and . These two entries will have different `MaxConnections`.
Therefore, when the xds server creates the upstream cluster, `foo`, it will randomly pick the `MaxConnections` from one of the entries.
**Possible solution:**
(i) If both `*` and `service name` appears in the ingress gateway listeners, shall the one with specific service name overwrite the one with "*"? Therefore, we can deterministically have one configuration for the upstream cluster.
@jkirschner-hashicorp , @mkeeler , WDYT?
### Consul info for both Client and Server
Main branch
Contributor guide
Assessment
This issue has not been assessed yet.