pingcap / pingcap/monitoring

Adjust the duration threshold for the alerts dependent on disk latency

Open
#19 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
29
Forks
46
Avg merge
12h 51m
Merged PRs (30d)
17

Description

Alerts that depend on disk latency are too sensitive for public cloud resources, and keeps triggering for each disk latency fluctuation. The following is the list of alerts that should be adjusted:

  • TiKV_async_request_write_duration_seconds
  • TiKV_scheduler_command_duration_seconds
  • TiKV_scheduler_latch_wait_duration_seconds

As per discussion with @tennix, we'll adjust the "for" clause from 1m to 5m for these alerts to prevent excessive false alarms in public cloud environment.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Search the repository for the three named alert definitions and inspect their current "for" clauses. Change each threshold from 1m to 5m, then validate the alert configuration and confirm all three alerts use the new duration.

Written by the indexing model from the issue text.

Assessment

Domain
observability
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.