prometheus-community / prometheus-community/stackdriver_exporter

Exporter fails when stackdriver updates their help strings

Open
#361 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
295
Forks
117
Avg merge
9d 14h
Merged PRs (30d)
7

Description

Recently the help string for some metrics changed, one such example metric is stackdriver_gce_instance_compute_googleapis_com_instance_uptime_total. This led to the exporter failing and yielding messages like the below (newlines added for readability)

34 error(s) occurred:
* [from Gatherer #2] collected metric stackdriver_gce_instance_compute_googleapis_com_instance_uptime_total label:{name:"instance_id" value:"REDACTED"} label:{name:"instance_name" value:"REDACTED"} label:{name:"project_id" value:"REDACTED"} label:{name:"unit" value:"s"} label:{name:"zone" value:"REDACTED"} gauge:{value:11520} timestamp_ms:1724192100000 

has help 

"Elapsed time since the VM was started, in seconds. After sampling, data is not visible for up to 120 seconds. When VM is Stopped (https://cloud.google.com/compute/docs/instances/stop-start-instance#stop-vm-google-cloud), the time is not calculated. On starting the VM again, the timer will reset to 0 for that VM." 

but should have 

"Elapsed time since the VM was started, in seconds."
... and many more of the same format

If this is something stackdriver is messing up I can open a Google support case instead, but this project or Google support seemed like the right place to handle it since the prometheus client does not allow changing these help strings (https://github.com/prometheus/client_golang/blob/b5361fed217651b4d855961b47481209ac0745a0/prometheus/registry.go#L640 causes the underlying failure)

I am not sure what the correct solution is here but I do know it is a really annoying issue since it means we essentially have to ignore groups of metrics at least until help string versions converge / age out. If there is a suggestion on how to fix this I can try to contribute as well, I just wasn't even sure where to start with this one.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reported stackdriver_gce_instance_compute_googleapis_com_instance_uptime_total failure and read the linked prometheus/client_golang registry.go validation around line 640. Trace how the changed Stackdriver help string reaches that validation and compare it with the expected text. Done means affected metric groups are gathered despite help-string updates instead of causing exporter errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
gcp, go, prometheus
Domain
observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.