GoogleCloudPlatform / GoogleCloudPlatform/k8s-stackdriver

Filtering metrics by labelSelector in external.metrics.k8s.io api doesn't work

Open
#535 1 comment 0 reactions 0 assignees View on GitHub
custom-metrics-stackdriver-adapter
Dominant language
Go
Stars
409
Forks
236
Avg merge
2h 34m
Merged PRs (30d)
9

Description

Some metrics labels (like `resource.labels.instance_id` in my case) are longer than 63 characters and have forbidden characters like slashes which makes filtering by those labels impossible and thus whole `v1beta1.external.metrics.k8s.io` service unusable for big number of external metrics.

If I run this with simpler labels like `resource.labels.region` I will get external metrics for all redis instances in the region as expected:
```
kubectl get --raw "/apis/external.metrics.k8s.io/v1beta1/namespaces/*/redis.googleapis.com|stats|network_traffic?labelSelector=resource.labels.region=europe-west3" | jq .

{
"kind": "ExternalMetricValueList",
"apiVersion": "external.metrics.k8s.io/v1beta1",
"metadata": {},
"items": [
{
"metricName": "redis.googleapis.com|stats|network_traffic",
"metricLabels": {
"metric.labels.direction": "in",
"metric.labels.role": "primary",
"resource.labels.instance_id": "projects/test-project/locations/europe-west3/instances/production",
"resource.labels.node_id": "node-0",
"resource.labels.project_id": "test-project",
"resource.labels.region": "europe-west3",
"resource.type": "redis_instance"
},
"timestamp": "2023-04-19T09:28:42Z",
"value": "3945867400m"
},
...
{
"metricName": "redis.googleapis.com|stats|network_traffic",
"metricLabels": {
"metric.labels.direction": "in",
"metric.labels.role": "primary",
"resource.labels.instance_id": "projects/test-project/locations/europe-west3/instances/staging",
"resource.labels.node_id": "node-0",
"resource.labels.project_id": "test-project",
"resource.labels.region": "europe-west3",
"resource.type": "redis_instance"
},
"timestamp": "2023-04-19T09:28:42Z",
"value": "13748666m"
},
]
}
```

Problem is if I want to select metric for specific redis instance using `resource.labels.instance_id` label that is not in [allowed format](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set) then I get error:
```
kubectl get --raw "/apis/external.metrics.k8s.io/v1beta1/namespaces/*/redis.googleapis.com|stats|network_traffic?labelSelector=resource.labels.instance_id=projects/test-project/locations/europe-west3/instances/staging" | jq .

Error from server (BadRequest): unable to parse requirement: values[0][resource.labels.instance_id]: Invalid value: "projects/test-project/locations/europe-west3/instances/staging": must be no more than 63 characters; a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue', or 'my_value', or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')
```

This makes horizontal pod autoscaling based on external metric impossible if you want to scale pods based on metrics from specific redis instance in the google cloud project.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.