4paradigm / 4paradigm/OpenMLDB

monitor issues

Open
#3,822 1 comment 0 reactions 1 assignee Claimed by @vagetablechicken View on GitHub
monitoring
Dominant language
C++
Stars
1.7k
Forks
331
Avg merge
12d 12h
Merged PRs (30d)
1

Description

origin https://github.com/vagetablechicken/fedb/issues/13

- taskmanager has no prometheus api, so don't add it in prometheus config
- / or /status print a html page, not good for prometheus, maybe openmldb-exporter can handle it, but info is simple.

- what means openmldb_connected_seconds_total? https://github.com/4paradigm/openmldb-exporter/blob/53792409f9a53cceaacf0091f658ca124692d356/openmldb_exporter/collector/collectors.py#L80
`current time - connect time in show component` incorrect: when openmldb-exporter is alive, stop clear start all, it can't reconnect to zk, and fail to execute show components or table status, won't update counters, so the openmldb_connected_seconds_total won't change. only zk restart can be handled, reset(start from small zxid) can't, check if needs improve
why grafana var role doesn't have tm? fix grafana dashboard template

- Table Partition Number show deleted tables partitions, we know it's deleted in graph, but the big number make me confused.
image

- openmldb_table_rows will up and down when insert data
image
test case is insert data into disk table

- two panel are the same metric, but different titile, confused
image

TODO all metrics in grafana, and how to read them when do perf profiling

# About doc
It's better to give a simple and clear steps for monitor deployment, current doc is hard to read.

## insert

When we want to know insert perf, we can get total time by client who do insertion.
DOING: kafka->openmldb perf test in openmldb-compose

# Grafana Update
- [ ] show taskmanager status in first row
- [ ] put metrics of tablets: we can know the insertion performance `xxx_openmldb_api_tablet_server_put[_xxx]`, latency and qps

new grafana dashboard:
https://gist.github.com/vagetablechicken/9428268b8cb8ba50d5867e6f0326da64

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.