hashicorp / hashicorp/consul

Local Rate limit definition is not working with Escape-Hatch Overrides configuration in Consul Envoy configuration

Open
#13,517 1 comment 0 reactions 0 assignees View on GitHub
theme/envoy/xds type/question
Dominant language
Go
Stars
30.1k
Forks
4.6k
Avg merge
1d 18h
Merged PRs (30d)
39

Description

Trying to registering sidecar into consul using hcl file and **consul services register** command with Local Rate limit definition inside filter level of Escape-Hatch Overrides configuration, throwing error following :
_[warning][config] [./source/common/config/grpc_stream.h:189] DeltaAggregatedResources gRPC config stream closed since 65898ms ago: 14, failed to generate all xDS resources from the snapshot: failed to generate xDS resources for "type.googleapis.com/envoy.config.listener.v3.Listener": can't unmarshal Any nested proto *envoy_config_listener_v3.Listener: can't unmarshal Any nested proto *envoy_extensions_filters_network_http_connection_manager_v3.HttpConnectionManager: unknown message type "envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit"_

**Definition:**
Try to implement rate-limiting [feature](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/local_rate_limit_filter#config-http-filters-local-rate-limit) of envoy which is running inside Consul infra. Since Consul have limited option to configure envoy we have chosen Escape-Hatch Overrides [configuration](https://www.consul.io/docs/connect/proxies/envoy#escape-hatch-overrides). We have defined all the config in hcl file and used that file to register the envoy sidecar service. Facing issue while updated with rate-limiting configuration inside filter level of Listener.

Tried three possible solution and none of the config is working, as following

1. Define LocalRateLimit filter inside http_filters of HttpConnectionManager [config-api-services.hcl](https://gist.github.com/subhajit-csc/d4e7b707e3b578c5547b4a90234e304a#file-config-api-services-hcl) . ### This case is applicable of inbound listener of sidecar
2. Define LocalRateLimit filter inside route level [config-api-services_with_route_level.hcl](https://gist.github.com/subhajit-csc/d4e7b707e3b578c5547b4a90234e304a#file-config-api-services_with_route_level-hcl). ### This case is applicable of inbound listener of sidecar
3. Tried to add same set of filter configuration in deferent field envoy_listener_json which is also another escape hatch configuration for outbound call [config-api-services_with_envoy_listener_json.hcl](https://gist.github.com/subhajit-csc/d4e7b707e3b578c5547b4a90234e304a#file-config-api-services_with_envoy_listener_json-hcl). ### In this case we want to limit all the call from sidecar to service via 8080 port.

#### Reproduction Steps

Steps to reproduce this issue, eg:
1. Run consul services register -token {{ service_token }} {{ sidecar_config_path }}/api_service.hcl
1. Run sidecar docker compose or docker run
1. Check sidecar docker container log

### Operating system and Environment details
Consul Version : 1.12.2
Envoy Version: envoy:v1.20.0

### Log Fragments
_storesidecar_1 | [2022-06-08 10:07:09.535][7][info][main] [source/server/server.cc:393] response trailer map: 160 bytes: grpc-message,grpc-status
storesidecar_1 | [2022-06-08 10:07:09.565][7][info][main] [source/server/server.cc:740] runtime: {}
storesidecar_1 | [2022-06-08 10:07:09.566][7][info][admin] [source/server/admin/admin.cc:135] admin address: 0.0.0.0:19000
storesidecar_1 | [2022-06-08 10:07:09.567][7][info][config] [source/server/configuration_impl.cc:127] loading tracing configuration
storesidecar_1 | [2022-06-08 10:07:09.567][7][info][config] [source/server/configuration_impl.cc:87] loading 0 static secret(s)
storesidecar_1 | [2022-06-08 10:07:09.567][7][info][config] [source/server/configuration_impl.cc:93] loading 4 cluster(s)
storesidecar_1 | [2022-06-08 10:07:09.754][7][info][config] [source/server/configuration_impl.cc:97] loading 2 listener(s)
storesidecar_1 | [2022-06-08 10:07:09.762][7][warning][misc] [source/common/protobuf/message_validator_impl.cc:21] Deprecated field: type envoy.extensions.filters.http.ext_authz.v3.ExtAuthz Using the default now-deprecated value AUTO for enum 'envoy.extensions.filters.http.ext_authz.v3.ExtAuthz.transport_api_version' from file ext_authz.proto. This enum value will be removed from Envoy soon so a non-default value must now be explicitly set. Please see https://www.envoyproxy.io/docs/envoy/latest/version_history/version_history for details. If continued use of this field is absolutely necessary, see https://www.envoyproxy.io/docs/envoy/latest/configuration/operations/runtime#using-runtime-overrides-for-deprecated-features for how to apply a temporary and highly discouraged override.
storesidecar_1 | [2022-06-08 10:07:09.762][7][warning][misc] [source/common/protobuf/message_validator_impl.cc:21] Deprecated field: type envoy.extensions.filters.http.ext_authz.v3.ExtAuthz Using the default now-deprecated value AUTO for enum 'envoy.extensions.filters.http.ext_authz.v3.ExtAuthz.transport_api_version' from file ext_authz.proto. This enum value will be removed from Envoy soon so a non-default value must now be explicitly set. Please see https://www.envoyproxy.io/docs/envoy/latest/version_history/version_history for details. If continued use of this field is absolutely necessary, see https://www.envoyproxy.io/docs/envoy/latest/configuration/operations/runtime#using-runtime-overrides-for-deprecated-features for how to apply a temporary and highly discouraged override.
storesidecar_1 | [2022-06-08 10:07:09.780][7][info][config] [source/server/configuration_impl.cc:109] loading stats configuration
storesidecar_1 | [2022-06-08 10:07:09.781][7][info][main] [source/server/server.cc:836] starting main dispatch loop
storesidecar_1 | [2022-06-08 10:07:09.791][7][info][runtime] [source/common/runtime/runtime_impl.cc:449] RTDS has finished initialization
storesidecar_1 | [2022-06-08 10:07:09.791][7][info][upstream] [source/common/upstream/cluster_manager_impl.cc:202] cm init: initializing cds
storesidecar_1 | [2022-06-08 10:07:09.792][7][warning][main] [source/server/server.cc:715] there is no configured limit to the number of allowed active connections. Set a limit via the runtime key overload.global_downstream_max_connections
storesidecar_1 | [2022-06-08 10:07:24.791][7][warning][config] [source/common/config/grpc_subscription_impl.cc:118] gRPC config: initial fetch timed out for type.googleapis.com/envoy.config.cluster.v3.Cluster
storesidecar_1 | [2022-06-08 10:07:24.791][7][info][upstream] [source/common/upstream/cluster_manager_impl.cc:206] cm init: all clusters initialized
storesidecar_1 | [2022-06-08 10:07:24.791][7][info][main] [source/server/server.cc:817] all clusters initialized. initializing init manager
storesidecar_1 | [2022-06-08 10:07:39.792][7][warning][config] [source/common/config/grpc_subscription_impl.cc:118] gRPC config: initial fetch timed out for type.googleapis.com/envoy.config.listener.v3.Listener
storesidecar_1 | [2022-06-08 10:07:39.792][7][info][config] [source/server/listener_manager_impl.cc:779] all dependencies initialized. starting workers
storesidecar_1 | [2022-06-08 10:07:50.605][7][warning][config] [./source/common/config/grpc_stream.h:189] DeltaAggregatedResources gRPC config stream closed since 40803ms ago: 14, failed to generate all xDS resources from the snapshot: failed to generate xDS resources for "type.googleapis.com/envoy.config.listener.v3.Listener": can't unmarshal Any nested proto *envoy_config_listener_v3.Listener: can't unmarshal Any nested proto *envoy_extensions_filters_network_http_connection_manager_v3.HttpConnectionManager: unknown message type "envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit"_

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with `consul services register` and the sidecar using the three referenced configurations: config-api-services.hcl, config-api-services_with_route_level.hcl, and config-api-services_with_envoy_listener_json.hcl. Start with the Envoy logs showing the unknown LocalRateLimit message type; done means the intended rate-limit configuration is accepted without the xDS listener generation error.

Written by the indexing model from the issue text.

Assessment

Domain
distributed-systems, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.