pingcap / pingcap/tidb-operator

tidb dashboard 监控指标采集问题

Open
#5,548 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
1.3k
Forks
540
Avg merge
3d 2h
Merged PRs (30d)
18

Description

Bug Report

What version of Kubernetes are you using?

k8s v1.27.8
tidb-cluster 7.5.0

What version of TiDB Operator are you using?

TiDB Operator Version: version.Info{GitVersion:"v1.5.2", GitCommit:"456a0273f67ac61212da78956f49f0a4a07e21d8", GitTreeState:"clean", BuildDate:"2024-01-19T03:50:22Z", GoVersion:"go1.21.5", Compiler:"gc", Platform:"linux/amd64"}
What storage classes exist in the Kubernetes cluster and what are used for PD/TiKV pods?

openEBS localPV

What's the status of the TiDB cluster pods?

What did you do?

What did you expect to see?

dashboard 有关 tidb 组件监控数据存在问题。

image

这个指标有问题,rate(process_cpu_seconds_total{job="tidb"}[120s])

通过 prometheus 查看,发现 job 不匹配。
image

dashboard grafana url 链接地址错误。

image

部分监控指标不显示,初步排查,发现都是 promQL 表达式有问题。
image

image

What did you see instead?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the dashboard panels and checking the Prometheus queries, including rate(process_cpu_seconds_total{job="tidb"}[120s]), against the reported job labels. Trace the dashboard configuration and Grafana URL referenced by the issue; done means the affected panels display data and the dashboard links and PromQL expressions work for the stated TiDB versions.

Written by the indexing model from the issue text.

Assessment

Tech stack
grafana, kubernetes, prometheus
Domain
observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.