envoyproxy / envoyproxy/gateway

Merge Gateway Prometheus Virtual Host metrics random assigned wrong with multiple listeners

Aperta
#3,305 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
help wanted
Lingua principale
Go
Stelle
3k
Fork
864
Merge medio
2g 2h
PR unite (30g)
140

Descrizione

*Description*:
Prometheus Virtual Host metrics are being assigned wrongly having cases where the wrong host is being attached to the wrong vhost.

I am not sure what the correct behaviour is in some cases we just get metrics like `envoy_vhost_vcluster_upstream_rq_retry` and in the current case we get `vhost..vcluster.` which is what the envoy docs say but with the wrong virtual host name attached to the wrong virtual host cluster.

A. The envoy_vhost_vcluster_upstream_rq_retry metrics that get emitted do have labels for `envoy_virtual_cluster` and `envoy_virtual_host`

B. `vhost..vcluster.` only have the `envoy_virtual_host` label attached and no cluster label.

I hope A is the correct way as its easier to build dashboards from.

*Repro steps*:

Take a example :

```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: foo
spec:
gatewayClassName: merge-gateway
listeners:
- name: foo.com
protocol: HTTP
hostname: foo.com
port: 80
allowedRoutes:
namespaces:
from: Same
- name: bar.com
protocol: HTTP
hostname: bar.com
port: 80
allowedRoutes:
namespaces:
from: Same
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: foobar-combined
spec:
parentRefs:
- name: foo
group: gateway.networking.k8s.io
kind: Gateway
hostnames:
- foo.com
- foobar.com
rules:
- backendRefs:
- kind: Service
group: ''
name: my-svc
port: 80
weight: 1
matches:
- path:
type: PathPrefix
value: /
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: a
spec:
gatewayClassName: merge-gateway
listeners:
- name: a.foo.com
protocol: HTTP
hostname: foo.com
port: 80
allowedRoutes:
namespaces:
from: Same
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: a
spec:
parentRefs:
- name: foo
group: gateway.networking.k8s.io
kind: Gateway
hostnames:
- a.foo.com
rules:
- backendRefs:
- kind: Service
group: ''
name: my-svc
port: 80
weight: 1
matches:
- path:
type: PathPrefix
value: /
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: ab
spec:
gatewayClassName: merge-gateway
listeners:
- name: ab.foo.com
protocol: HTTP
hostname: ab.foo.com
port: 80
allowedRoutes:
namespaces:
from: Same
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: foobar-combined
spec:
parentRefs:
- name: ab
group: gateway.networking.k8s.io
kind: Gateway
hostnames:
- ab.foo.com
rules:
- backendRefs:
- kind: Service
group: ''
name: my-svc
port: 80
weight: 1
matches:
- path:
type: PathPrefix
value: /
```

Its hard to pinpoint what's going on and seems to be random in how it picks for example I got ```envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_retry{envoy_virtual_host="api-gateway/ab/ab"} 0```
for the metric name.

Metrics dump
```
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_retry counter
envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_retry{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_retry_limit_exceeded counter
envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_retry_limit_exceeded{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_retry_overflow counter
envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_retry_overflow{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_retry_success counter
envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_retry_success{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_timeout counter
envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_timeout{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_total counter
envoy_vhost_foo_com_ab_foo_com_vcluster_ab_foo_com_upstream_rq_total{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_retry counter
envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_retry{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_retry_limit_exceeded counter
envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_retry_limit_exceeded{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_retry_overflow counter
envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_retry_overflow{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_retry_success counter
envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_retry_success{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_timeout counter
envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_timeout{envoy_virtual_host="api-gateway/ab/ab"} 0
# TYPE envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_total counter
envoy_vhost_foo_com_ab_foo_com_vcluster_other_upstream_rq_total{envoy_virtual_host="api-gateway/ab/ab"} 0
```

it looks like its merging the virtual clusters.

*Environment*:
k8s 1.29.0
envoy gateway 1.0.1 Merge gateways

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.