hashicorp / hashicorp/consul

Consul global proxy configuration not propagating to the proxies

Open
#21,986 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Go
Stars
30.1k
Forks
4.6k
Avg merge
2d 6h
Merged PRs (30d)
43

Description

### Consul global proxy configuration not propagating to the envoy proxies configured via consul connect:

I've a service mesh on ECS with EC2 working as the control plane where my consul server is installed and configured correctly. All my services on ECS work with a consul agent that redirects the requests within the mesh and each service as Envoy working as proxy to communicate with the consul agent (Envoy is installed as a sidecar container for each service). The proxies are configured using the consul connect command: `consul connect envoy`

My issue is when I apply a global configuration to enable access logging on my Envoy proxies, as in: https://developer.hashicorp.com/consul/docs/connect/observability/access-logs the proxies do not have logs and appear not to have the configuration enabled. Is there something I'm missing?

### Versions and details:

Consul: 1.18.2
Envoy: 1.28.7
Services: ECS
Consul Leader: EC2 instance with 1 leader and 2 followers

Output of `consul config read -kind proxy-defaults -name global`:
```
{
"Kind": "proxy-defaults",
"Name": "global",
"TransparentProxy": {},
"Config": {
"LocalRequestTimeoutMs": 15000,
"Protocol": "http"
},
"MeshGateway": {},
"Expose": {},
"AccessLogs": {
"Enabled": true,
"JSONFormat": "{\n \"timestamp\": \"%START_TIME%\",\n \"protocol\": \"%PROTOCOL%\",\n \"duration\": \"%DURATION%\",\n \"upstream_host\": \"%UPSTREAM_HOST%\",\n \"upstream_cluster\": \"%UPSTREAM_CLUSTER%\",\n \"response_code\": \"%RESPONSE_CODE%\",\n \"response_flags\": \"%RESPONSE_FLAGS%\",\n \"bytes_received\": \"%BYTES_RECEIVED%\",\n \"bytes_sent\": \"%BYTES_SENT%\",\n \"request_method\": \"%REQ(:METHOD)%\",\n \"request_path\": \"%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%\",\n \"request_id\": \"%REQ(X-REQUEST-ID)%\",\n \"request_host\": \"%REQ(HOST)%\",\n \"user_agent\": \"%REQ(USER-AGENT)%\",\n \"client_ip\": \"%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%\",\n \"connection_termination_details\": \"%CONNECTION_TERMINATION_DETAILS%\",\n \"upstream_transport_failure_reason\": \"%UPSTREAM_TRANSPORT_FAILURE_REASON%\"\n}\n"
},
"CreateIndex": 3666562,
"ModifyIndex": 17581039
}
```

### Additional information:

After writing the global configuration I checked that all the consul server instances have it correctly applied, I also restarted the services to bootstrap them from scratch. Although nothing seems to work. On the Envoy proxies, looking at `curl localhost:19000/config_dump` the access logging is not enabled for the sidecar proxies.

Output sample of the Envoy config_dump:
```
"dynamic_listeners": [
{
"name": "public_listener:[REDACTED_IP]:[REDACTED_PORT]",
"active_state": {
"version_info": "[REDACTED]",
"listener": {
"@type": "type.googleapis.com/envoy.config.listener.v3.Listener",
"name": "public_listener:[REDACTED_IP]:[REDACTED_PORT]",
"address": {
"socket_address": {
"address": "[REDACTED_IP]",
"port_value": [REDACTED_PORT]
}
},
"filter_chains": [
{
"filters": [
{
"name": "envoy.filters.network.rbac",
"typed_config": {
"@type": "type.googleapis.com/envoy.extensions.filters.network.rbac.v3.RBAC",
"rules": {
"policies": {
"consul-intentions-layer4": {
"permissions": [
{
"any": true
}
],
"principals": [
{
"authenticated": {
"principal_name": {
"safe_regex": {
"regex": "^spiffe://[REDACTED]"
}
}
}
}
]
}
}
},
"stat_prefix": "connect_authz"
}
},
{
"name": "envoy.filters.network.tcp_proxy",
"typed_config": {
"@type": "type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy",
"stat_prefix": "public_listener",
"cluster": "local_app"
}
}
],
"transport_socket": {
"name": "tls",
"typed_config": {
"@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext",
"common_tls_context": {
"tls_params": {},
"tls_certificates": [
{
"certificate_chain": {
"inline_string": "[REDACTED_CERTIFICATE]"
},
"private_key": {
"inline_string": "[REDACTED]"
}
}
],
"validation_context": {
"trusted_ca": {
"inline_string": "[REDACTED_CA_CERTIFICATE]"
}
}
},
"require_client_certificate": true
}
}
}
],
"traffic_direction": "INBOUND"
},
"last_updated": "[REDACTED_TIMESTAMP]"
}
},
{
"name": "mesh-sb-test-appliB:[REDACTED_IP]:[REDACTED_PORT]",
"active_state": {
"version_info": "[REDACTED]",
"listener": {
"@type": "type.googleapis.com/envoy.config.listener.v3.Listener",
"name": "mesh-sb-test-appliB:[REDACTED_IP]:[REDACTED_PORT]",
"address": {
"socket_address": {
"address": "[REDACTED_IP]",
"port_value": [REDACTED_PORT]
}
},
"filter_chains": [
{
"filters": [
{
"name": "envoy.filters.network.http_connection_manager",
"typed_config": {
"@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager",
"stat_prefix": "[REDACTED]",
"route_config": {
"name": "[REDACTED]",
"virtual_hosts": [
{
"name": "[REDACTED]",
"domains": [
"*"
],
"routes": [
{
"match": {
"prefix": "/"
},
"route": {
"cluster": "[REDACTED]"
}
}
]
}
]
},
"http_filters": [
{
"name": "envoy.filters.http.router",
"typed_config": {
"@type": "type.googleapis.com/envoy.extensions.filters.http.router.v3.Router"
}
}
],
"tracing": {
"random_sampling": {}
},
"upgrade_configs": [
{
"upgrade_type": "websocket"
}
]
}
}
]
}
],
"traffic_direction": "OUTBOUND"
},
"last_updated": "[REDACTED_TIMESTAMP]"
}
}
]
```

Let me know if you need more information.

Kind Regards,

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.