envoyproxy / envoyproxy/gateway

Merge Gateway Prometheus Virtual Host metrics random assigned wrong with multiple listeners

Abierto
#3,305 1 comentario 0 reacciones 0 asignados Ver en GitHub
help wanted
Lenguaje dominante
Go
Estrellas
3k
Forks
864
Merge medio
2 d 2 h
PR fusionados (30 d)
140

Descripción

*Description*:
Prometheus Virtual Host metrics are being assigned wrongly having cases where the wrong host is being attached to the wrong vhost.

I am not sure what the correct behaviour is in some cases we just get metrics like `envoy_vhost_vcluster_upstream_rq_retry` and in the current case we get `vhost..vcluster.` which is what the envoy docs say but with the wrong virtual host name attached to the wrong virtual host cluster.

A. The envoy_vhost_vcluster_upstream_rq_retry metrics that get emitted do have labels for `envoy_virtual_cluster` and `envoy_virtual_host`

B. `vhost..vcluster.` only have the `envoy_virtual_host` label attached and no cluster label.

I hope A is the correct way as its easier to build dashboards from.

*Repro steps*:

Take a example :

```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: foo
spec:
gatewayClassName: merge-gateway
listeners:
- name: foo.com
protocol: HTTP
hostname: foo.com
port: 80
allowedRoutes:
namespaces:
from: Same
- name: bar.com
protocol: HTTP
hostname: bar.com
port: 80
allowedRoutes:
namespaces:
from: Same
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: foobar-combined
spec:
parentRefs:
- name: foo
group: gateway.networking.k8s.io
kind: Gateway
hostnames:
- foo.com
- foobar.com
rules:
- backendRefs:
- kind: Service
group: ''
name: my-svc
port: 80
weight: 1
matches:
- path:
type: PathPrefix
value: /
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: a
spec:
gatewayClassName: merge-gateway
listeners:
- name: a.foo.com
protocol: HTTP
hostname: foo.com
port: 80
allowedRoutes:
namespaces:
from: Same
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: a
spec:
parentRefs:
- name: foo
group: gateway.networking.k8s.io
kind: Gateway
hostnames:
- a.foo.com
rules:
- backendRefs:
- kind: Service
group: ''
name: my-svc
port: 80
weight: 1
matches:
- path:
type: PathPrefix
value: /
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: ab
spec:
gatewayClassName: merge-gateway
listeners:
- name: ab.foo.com
protocol: HTTP
hostname: ab.foo.com
port: 80
allowedRoutes:
namespaces:
from: Same
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: foobar-combined
spec:
parentRefs:
- name: ab
group: gateway.networking.k8s.io
kind: Gateway
hostnames:
- ab.foo.com
rules:
- backendRefs:
- kind: Service
group: ''
name: my-svc
port: 80
weight: 1
matches:
- path:
type: PathPrefix
value: /
```

Its hard to pinpoint what's going on and seems to be random in how it picks for example I got ```envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_retry{envoy_virtual_host="api-gateway/ab/ab"} 0```
for the metric name.

Metrics dump
```
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_retry counter
envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_retry{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_retry_limit_exceeded counter
envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_retry_limit_exceeded{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_retry_overflow counter
envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_retry_overflow{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_retry_success counter
envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_retry_success{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_timeout counter
envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_timeout{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_total counter
envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_total{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_retry counter
envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_retry{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_retry_limit_exceeded counter
envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_retry_limit_exceeded{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_retry_overflow counter
envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_retry_overflow{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_retry_success counter
envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_retry_success{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_timeout counter
envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_timeout{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_total counter
envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_total{envoy_virtual_host="api-gateway/ab/ab"} 0
```

it looks like its merging the virtual clusters.

*Environment*:
k8s 1.29.0
envoy gateway 1.0.1 Merge gateways

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.