tikv / tikv/pd

cluster health status not working

Open
#3,077 6 comments 0 reactions 1 assignee Claimed by @nolouch View on GitHub
type/question
Dominant language
Go
Stars
1.2k
Forks
783
Avg merge
5d 21h
Merged PRs (30d)
36

Description

## Bug Report

In #2964 I commented on the cluster health status metric (pd_cluster_health_status) having an issue w/ stale members.
The resolution also corrected the inverted boolean sense (so now 1 == good, 0 == bad).

However, I now in v4.0.7 see two issues:

1. the metric is only available from one of the pd (the other pd in the cluster have no metric called `pd_cluster_health_status`)
2. the value still shows 0

so e.g;

```
curl http://pd-0:2379 |grep health_status -> no results
curl http://pd-1:2379 | grep health_status -> no results
curl http://pd-2:2379 | grep health_status
# HELP pd_cluster_health_status Status of the cluster.
# TYPE pd_cluster_health_status gauge
pd_cluster_health_status{name="db-pd-0"} 0
pd_cluster_health_status{name="db-pd-1"} 0
pd_cluster_health_status{name="db-pd-2"} 0
```

a) why the 0?
b) why does db-0 and db-1 not respond?

this is making it impossible to construct a prometheus alert

### What did you do?

### What did you expect to see?

i expect that:
a) all pd show cluster_health_status metric
b) the result code is 1 for good, 0 for bad (or at least document it?)

### What did you see instead?

### What version of PD are you using (`pd-server -V`)?

v4.0.7

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.