hashicorp / hashicorp/consul

`consul_version` metric does not work properly with Prometheus

Open
#13,498 1 comment 7 reactions 0 assignees View on GitHub
theme/telemetry type/enhancement
Dominant language
Go
Stars
30.1k
Forks
4.6k
Avg merge
1d 18h
Merged PRs (30d)
39

Description

The `consul_version` metric does not work properly with Prometheus.

It is set here: https://github.com/hashicorp/consul/blob/a407d378af120741ef9fa8745bc54fa2db56644f/agent/agent.go#L754-L758

However since it is set only once at agent start, it expires and is no longer reported once `prometheus_retention_time` elapses.

Note that after it expires, the dummy `consul_version` metric without any labels will continue to be reported with value zero (the dummy metrics are further discussed in #13495), which may add to the confusion of a Consul operator attempting to use the metric.

Fixing this properly may be challenging, because of the design of the `armon/go-metrics` API.

Contributor guide

Open the contributing guide

Research direction

Start in agent/agent.go at the consul_version setup around lines 754-758, then read the armon/go-metrics API behavior for Prometheus retention. Reproduce the metric disappearing after prometheus_retention_time elapses and account for the unlabeled dummy metric described in #13495. Done means the labeled consul_version metric remains usable to Prometheus after retention.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, prometheus
Domain
backend, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.