hashicorp / hashicorp/consul

Incorrect metrics: consul_server_isLeader

Open
#18,616 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
30.1k
Forks
4.6k
Avg merge
1d 18h
Merged PRs (30d)
39

Description

#### Overview of the Issue
The consul_server_isLeader in metrics endpoint is always `0` (follower), never `1` (leader).

---

#### Reproduction Steps
First, find the leader
```
curl -sv http://:8500/v1/status/leader
```

Enable the metrics endpoint and hit the leader node like this:
```
> curl -sv http://:8500/v1/agent/metrics?format=prometheus | grep consul_server_isLeader
...
# HELP consul_server_isLeader Tracks if the server is a leader.
# TYPE consul_server_isLeader gauge
consul_server_isLeader 0
```
It should be `1` not `0`.

### Consul info for both Client and Server

Server info

```
agent:
check_monitors = 0
check_ttls = 0
checks = 0
services = 0
build:
prerelease =
revision = 0ba7a401
version = 1.14.2
version_metadata =
consul:
acl = disabled
bootstrap = false
known_datacenters = 1
leader = false
leader_addr = <... sensitive ....>
server = true
raft:
applied_index = 124936906
commit_index = 124936906
fsm_pending = 0
last_contact = 3.636925ms
last_log_index = 124936906
last_log_term = 22
last_snapshot_index = 124936850
last_snapshot_term = 20
latest_configuration = <... sensitive ....>
latest_configuration_index = 0
num_peers = 2
protocol_version = 3
protocol_version_max = 3
protocol_version_min = 0
snapshot_version_max = 1
snapshot_version_min = 0
state = Follower
term = 22
runtime:
arch = amd64
cpu_count = 8
goroutines = 146
max_procs = 8
os = linux
version = go1.19.2
serf_lan:
coordinate_resets = 0
encrypted = false
event_queue = 0
event_time = 10
failed = 0
health_score = 0
intent_queue = 0
left = 0
member_time = 10514
members = 16
query_queue = 0
query_time = 2
serf_wan:
coordinate_resets = 0
encrypted = false
event_queue = 0
event_time = 1
failed = 0
health_score = 0
intent_queue = 0
left = 0
member_time = 13
members = 3
query_queue = 0
query_time = 1
```

### Operating system and Environment details

K8S with the official helm chart.

### Log Fragments

N/A

Contributor guide

Open the contributing guide

Research direction

Start at the /v1/agent/metrics endpoint and trace how the consul_server_isLeader gauge derives its value from the server's leadership state. Reproduce with the provided curl command against both leader and follower nodes, then verify that the metric reports 1 for the leader and 0 for followers.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, prometheus
Domain
observability
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.