prometheus / prometheus/alertmanager

Send "New Silence" notification (trigger or list of fingerprints)

Open
#2,690 1 comment 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

component/notify stale
Dominant language
Go
Stars
8.6k
Forks
2.5k
Avg merge
2d 6h
Merged PRs (30d)
61

Description

Could be useful to have the possibility of configuring get a notification when a new silence is created.
I spend a reasonable amount of time reading other issues, documentation, etc, and I couldn't find anything.

My use case

I'm deploying a new Prometheus and Alertmanager stack with notifications to Google Chat, to accomplish that currently i'm using mr-karan/calert. Meanwhile I'm developing (in my free time, just started) a Node.js proxy to send the notifications to the chat application by REST API.
The idea it's to grouping the alerts by the fingerprint to map them in a specific thread. I'm thinking of making a command in the bot, to create a new silence if it's called from a specific "alert thread", for example @Alertmanager silence

Given this, my idea it's to have a field in global configuration (or in the receivers), that allows send notification triggered by a new Silence created (maybe at expiration too), with the same fingerprint of the alert silenced (only if it available all the information to make the
required hash).
Another idea, a little bit easier I thinks, it's have the possibility of have custom fields in the silences or at least a list of fingerprints associated to the silence. Having this we could identify from the CLI, API etc, without having a trigger to accomplish this.

Other ideas, suggestions and further information in the configurations are welcome.
Regards.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no repository files or tests. Start by tracing silence creation through the CLI/API and the global configuration or receivers, then compare existing notification paths. Done would require a settled design and tests for the chosen new-silence notification, fingerprint list, or custom-field behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, prometheus
Domain
observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.