prometheus-community / prometheus-community/stackdriver_exporter
Apigee metrics from GCP - bucket type prometheus metrics value is not increasing monotonically
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 295
- Forks
- 117
- Avg merge
- 9d 14h
- Merged PRs (30d)
- 7
Description
We've been introduced to this stackdriver_exporter recently and started using it to export metrics from GCP infra to our prometheus. Specifically the metrics of Apigee proxy provided as a managed service in GCP. We found a specific type of metric which got exported through the stackdriver exporter is not increasing monotonically in its value over the time. This metric type is used to bucket the range of values in different buckets which then can be used to calculate quantiles using histogram_quantile function with prometheus.
Here is an example for metric which is used for calculating latency whose value didn't increase monotonically
Error got from prometheus while calculating histogram quantile using the above metric
This is a helm based deployment in a kubernetes cluster.
Helm Chart version deployed - v4.5.0
Exporter app version deployed - v0.15.0
Expected:
The metric _bucket value has to be increasing always like a counter type metric
Help needed . Thanks
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No source file or test is identified in the report. Start by reproducing the Apigee _bucket metric behavior with Helm chart v4.5.0 and exporter v0.15.0, then trace how the metric is exported and compare it with Prometheus histogram expectations. Done means the bucket values increase monotonically and histogram_quantile no longer reports the shown error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- gcp, go, helm, kubernetes, prometheus
- Domain
- cloud, infrastructure, observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100