GoogleCloudPlatform / GoogleCloudPlatform/k8s-stackdriver

custom-metrics-stackdriver-adapter not working - auth problems?

Open
#498 13 comments 10 reactions 0 assignees View on GitHub
custom-metrics-stackdriver-adapter
Dominant language
Go
Stars
409
Forks
236
Avg merge
2h 34m
Merged PRs (30d)
9

Description

Hi have created a new GCP project and a GKE cluster inside it.

I've followed the instructions in the [README](https://github.com/GoogleCloudPlatform/k8s-stackdriver/blob/master/custom-metrics-stackdriver-adapter/README.md)...

However Logs Explorer is complaining about it... looks like an authentication problem?

```
{
"insertId": "0zumoy6o5n39r4ul",
"jsonPayload": {
"pid": "1",
"message": "apiserver received an error that is not an metav1.Status: &errors.errorString{s:\"http2: stream closed\"}: http2: stream closed"
},
"resource": {
"type": "k8s_container",
"labels": {
"pod_name": "custom-metrics-stackdriver-adapter-848d58c7c7-h4s5d",
"cluster_name": "scheduled-autoscaler",
"namespace_name": "custom-metrics",
"container_name": "pod-custom-metrics-stackdriver-adapter",
"location": "us-central1-f",
"project_id": "rbarbero-gke-freshness-3"
}
},
"timestamp": "2022-11-23T20:10:54.385935713Z",
"severity": "ERROR",
"labels": {
"compute.googleapis.com/resource_name": "gke-scheduled-autoscaler-default-pool-28d0994f-85hh",
"k8s-pod/run": "custom-metrics-stackdriver-adapter",
"k8s-pod/k8s-app": "custom-metrics-stackdriver-adapter",
"k8s-pod/pod-template-hash": "848d58c7c7",
"k8s-pod/kubernetes_io/cluster-service": "true"
},
"logName": "projects/rbarbero-gke-freshness-3/logs/stderr",
"sourceLocation": {
"file": "status.go",
"line": "71"
},
"receiveTimestamp": "2022-11-23T20:10:57.325326887Z"
},
{
"insertId": "7crfx9uy4t13qxty",
"jsonPayload": {
"pid": "1",
"message": "post-timeout activity - time-elapsed: 108.975022ms, GET \"/apis/custom.metrics.k8s.io/v1beta1\" result: "
},
"resource": {
"type": "k8s_container",
"labels": {
"container_name": "pod-custom-metrics-stackdriver-adapter",
"cluster_name": "scheduled-autoscaler",
"pod_name": "custom-metrics-stackdriver-adapter-848d58c7c7-h4s5d",
"location": "us-central1-f",
"project_id": "rbarbero-gke-freshness-3",
"namespace_name": "custom-metrics"
}
},
"timestamp": "2022-11-23T20:10:54.389245350Z",
"severity": "ERROR",
"labels": {
"k8s-pod/run": "custom-metrics-stackdriver-adapter",
"k8s-pod/k8s-app": "custom-metrics-stackdriver-adapter",
"k8s-pod/pod-template-hash": "848d58c7c7",
"k8s-pod/kubernetes_io/cluster-service": "true",
"compute.googleapis.com/resource_name": "gke-scheduled-autoscaler-default-pool-28d0994f-85hh"
},
"logName": "projects/rbarbero-gke-freshness-3/logs/stderr",
"sourceLocation": {
"file": "timeout.go",
"line": "135"
},
"receiveTimestamp": "2022-11-23T20:10:57.325326887Z"
},
{
"insertId": "484cm7yu3ea24ke2",
"jsonPayload": {
"message": "apiserver was unable to write a fallback JSON response: http2: stream closed",
"pid": "1"
},
"resource": {
"type": "k8s_container",
"labels": {
"pod_name": "custom-metrics-stackdriver-adapter-848d58c7c7-h4s5d",
"location": "us-central1-f",
"namespace_name": "custom-metrics",
"container_name": "pod-custom-metrics-stackdriver-adapter",
"project_id": "rbarbero-gke-freshness-3",
"cluster_name": "scheduled-autoscaler"
}
},
"timestamp": "2022-11-23T20:10:54.391067637Z",
"severity": "ERROR",
"labels": {
"k8s-pod/pod-template-hash": "848d58c7c7",
"k8s-pod/run": "custom-metrics-stackdriver-adapter",
"k8s-pod/k8s-app": "custom-metrics-stackdriver-adapter",
"k8s-pod/kubernetes_io/cluster-service": "true",
"compute.googleapis.com/resource_name": "gke-scheduled-autoscaler-default-pool-28d0994f-85hh"
},
"logName": "projects/rbarbero-gke-freshness-3/logs/stderr",
"sourceLocation": {
"file": "writers.go",
"line": "130"
},
"receiveTimestamp": "2022-11-23T20:10:57.325326887Z"
},
{
"insertId": "vku7ud5xhug364zm",
"jsonPayload": {
"message": "post-timeout activity - time-elapsed: 193.853136ms, GET \"/apis/custom.metrics.k8s.io/v1beta1\" result: ",
"pid": "1"
},
"resource": {
"type": "k8s_container",
"labels": {
"namespace_name": "custom-metrics",
"location": "us-central1-f",
"container_name": "pod-custom-metrics-stackdriver-adapter",
"cluster_name": "scheduled-autoscaler",
"pod_name": "custom-metrics-stackdriver-adapter-848d58c7c7-h4s5d",
"project_id": "rbarbero-gke-freshness-3"
}
},
"timestamp": "2022-11-23T20:10:54.486030132Z",
"severity": "ERROR",
"labels": {
"k8s-pod/kubernetes_io/cluster-service": "true",
"k8s-pod/k8s-app": "custom-metrics-stackdriver-adapter",
"k8s-pod/run": "custom-metrics-stackdriver-adapter",
"compute.googleapis.com/resource_name": "gke-scheduled-autoscaler-default-pool-28d0994f-85hh",
"k8s-pod/pod-template-hash": "848d58c7c7"
},
"logName": "projects/rbarbero-gke-freshness-3/logs/stderr",
"sourceLocation": {
"file": "timeout.go",
"line": "135"
},
"receiveTimestamp": "2022-11-23T20:10:57.325326887Z"
},
{
"insertId": "u55i6pz9rfhil94j",
"jsonPayload": {
"message": "timeout or abort while handling: method=GET URI=\"/apis/custom.metrics.k8s.io/v1beta1\" audit-ID=\"3e30bad8-85bc-46e4-9a9b-0680dbfe47f8\"",
"pid": "1"
},
"resource": {
"type": "k8s_container",
"labels": {
"container_name": "pod-custom-metrics-stackdriver-adapter",
"location": "us-central1-f",
"cluster_name": "scheduled-autoscaler",
"namespace_name": "custom-metrics",
"project_id": "rbarbero-gke-freshness-3",
"pod_name": "custom-metrics-stackdriver-adapter-848d58c7c7-h4s5d"
}
},
"timestamp": "2022-11-23T20:10:54.502749250Z",
"severity": "ERROR",
"labels": {
"k8s-pod/run": "custom-metrics-stackdriver-adapter",
"k8s-pod/pod-template-hash": "848d58c7c7",
"k8s-pod/kubernetes_io/cluster-service": "true",
"compute.googleapis.com/resource_name": "gke-scheduled-autoscaler-default-pool-28d0994f-85hh",
"k8s-pod/k8s-app": "custom-metrics-stackdriver-adapter"
},
"logName": "projects/rbarbero-gke-freshness-3/logs/stderr",
"sourceLocation": {
"file": "wrap.go",
"line": "54"
},
"receiveTimestamp": "2022-11-23T20:10:57.325326887Z"
},
{
"insertId": "foryseqqz48yqfbg",
"jsonPayload": {
"pid": "1",
"message": "apiserver was unable to close cleanly the response writer: http: Handler timeout"
},
"resource": {
"type": "k8s_container",
"labels": {
"project_id": "rbarbero-gke-freshness-3",
"pod_name": "custom-metrics-stackdriver-adapter-848d58c7c7-h4s5d",
"namespace_name": "custom-metrics",
"cluster_name": "scheduled-autoscaler",
"location": "us-central1-f",
"container_name": "pod-custom-metrics-stackdriver-adapter"
}
},
"timestamp": "2022-11-23T20:10:54.502797138Z",
"severity": "ERROR",
"labels": {
"compute.googleapis.com/resource_name": "gke-scheduled-autoscaler-default-pool-28d0994f-85hh",
"k8s-pod/kubernetes_io/cluster-service": "true",
"k8s-pod/run": "custom-metrics-stackdriver-adapter",
"k8s-pod/k8s-app": "custom-metrics-stackdriver-adapter",
"k8s-pod/pod-template-hash": "848d58c7c7"
},
"logName": "projects/rbarbero-gke-freshness-3/logs/stderr",
"sourceLocation": {
"file": "writers.go",
"line": "111"
},
"receiveTimestamp": "2022-11-23T20:10:57.325326887Z"
},
{
"insertId": "ic9b2l6ofwbvvdx0",
"jsonPayload": {
"message": "post-timeout activity - time-elapsed: 4.947211ms, GET \"/apis/custom.metrics.k8s.io/v1beta1\" result: ",
"pid": "1"
},
"resource": {
"type": "k8s_container",
"labels": {
"pod_name": "custom-metrics-stackdriver-adapter-848d58c7c7-h4s5d",
"project_id": "rbarbero-gke-freshness-3",
"namespace_name": "custom-metrics",
"location": "us-central1-f",
"cluster_name": "scheduled-autoscaler",
"container_name": "pod-custom-metrics-stackdriver-adapter"
}
},
"timestamp": "2022-11-23T20:10:54.502805932Z",
"severity": "ERROR",
"labels": {
"k8s-pod/run": "custom-metrics-stackdriver-adapter",
"compute.googleapis.com/resource_name": "gke-scheduled-autoscaler-default-pool-28d0994f-85hh",
"k8s-pod/pod-template-hash": "848d58c7c7",
"k8s-pod/k8s-app": "custom-metrics-stackdriver-adapter",
"k8s-pod/kubernetes_io/cluster-service": "true"
},
"logName": "projects/rbarbero-gke-freshness-3/logs/stderr",
"sourceLocation": {
"file": "timeout.go",
"line": "135"
},
"receiveTimestamp": "2022-11-23T20:10:57.325326887Z"
}
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.