GoogleCloudPlatform / GoogleCloudPlatform/k8s-stackdriver

How to modify the default prometheus-to-sd resource limits?

Open
#327 10 comments 1 reaction 0 assignees View on GitHub
Dominant language
Go
Stars
409
Forks
236
Avg merge
2h 34m
Merged PRs (30d)
9

Description

Hi,

Several of my GKE clusters experience constant CPUThrottlingHigh alerts, coming from prometheus-to-sd pods.

This deployment has incredibly low CPU requests/limits by default:

resources:
limits:
cpu: 3m
memory: 20Mi
requests:
cpu: 1m
memory: 20Mi

When I try to edit the prometheus-to-sd daemonset and increase these values, they get reverted to defaults.

Questions:

1) Is it possible to modify these default values in any way?

2) Why are they so low? Such low values are very likely to cause CPU throttling and increase the monitoring noise from GKE clusters.

Similar report by a different user: https://stackoverflow.com/questions/58182345/cpu-throttling-on-default-gke-pods

My GKE cluster version is v1.15.11-gke.5

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.