GoogleCloudPlatform / GoogleCloudPlatform/k8s-stackdriver
Filtering metrics by labelSelector in external.metrics.k8s.io api doesn't work
- Dominant language
- Go
- Stars
- 409
- Forks
- 236
- Avg merge
- 2h 34m
- Merged PRs (30d)
- 9
Description
Some metrics labels (like `resource.labels.instance_id` in my case) are longer than 63 characters and have forbidden characters like slashes which makes filtering by those labels impossible and thus whole `v1beta1.external.metrics.k8s.io` service unusable for big number of external metrics.
If I run this with simpler labels like `resource.labels.region` I will get external metrics for all redis instances in the region as expected:
```
kubectl get --raw "/apis/external.metrics.k8s.io/v1beta1/namespaces/*/redis.googleapis.com|stats|network_traffic?labelSelector=resource.labels.region=europe-west3" | jq .
{
"kind": "ExternalMetricValueList",
"apiVersion": "external.metrics.k8s.io/v1beta1",
"metadata": {},
"items": [
{
"metricName": "redis.googleapis.com|stats|network_traffic",
"metricLabels": {
"metric.labels.direction": "in",
"metric.labels.role": "primary",
"resource.labels.instance_id": "projects/test-project/locations/europe-west3/instances/production",
"resource.labels.node_id": "node-0",
"resource.labels.project_id": "test-project",
"resource.labels.region": "europe-west3",
"resource.type": "redis_instance"
},
"timestamp": "2023-04-19T09:28:42Z",
"value": "3945867400m"
},
...
{
"metricName": "redis.googleapis.com|stats|network_traffic",
"metricLabels": {
"metric.labels.direction": "in",
"metric.labels.role": "primary",
"resource.labels.instance_id": "projects/test-project/locations/europe-west3/instances/staging",
"resource.labels.node_id": "node-0",
"resource.labels.project_id": "test-project",
"resource.labels.region": "europe-west3",
"resource.type": "redis_instance"
},
"timestamp": "2023-04-19T09:28:42Z",
"value": "13748666m"
},
]
}
```
Problem is if I want to select metric for specific redis instance using `resource.labels.instance_id` label that is not in [allowed format](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set) then I get error:
```
kubectl get --raw "/apis/external.metrics.k8s.io/v1beta1/namespaces/*/redis.googleapis.com|stats|network_traffic?labelSelector=resource.labels.instance_id=projects/test-project/locations/europe-west3/instances/staging" | jq .
Error from server (BadRequest): unable to parse requirement: values[0][resource.labels.instance_id]: Invalid value: "projects/test-project/locations/europe-west3/instances/staging": must be no more than 63 characters; a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue', or 'my_value', or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')
```
This makes horizontal pod autoscaling based on external metric impossible if you want to scale pods based on metrics from specific redis instance in the google cloud project.
Contributor guide
Assessment
This issue has not been assessed yet.