prometheus / prometheus/alertmanager
Group Timings Visibility
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 8.6k
- Forks
- 2.5k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 61
Description
When designing an alerting pipeline, it's important for an operator to understand how the semantics of the system map their understanding.
The Alertmanager has some knobs such as:
- Group Interval (Global and routing specific)
- Repeat Interval (Global and routing specific)
- Group Wait (Global and routing specific)
- HA_peer timeout (HA mode only + per replica)
Currently, there are no metrics or visibility into any of these - so operators are left on "waiting" to see what happens or checking to see if any of the log lines make sense.
I think we can do better and provide visibility into the current state of all groups, so that operators can answer questions such as:
- What this is alert group was last evaluated?
- What state are these alert groups that I see?
- Which replica is firing what?
- What is the state of this replica?
I recognise that these answers are not simple when we think about alertmanager in HA mode and there's a chance of divergence but I think we can do better than what we currently do at the moment.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are named. Start by locating how group timing and HA state are managed, then define the visibility needed for group evaluation, replica state, and waiting intervals; done requires an agreed scope and tests for the proposed behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100