prometheus / prometheus/alertmanager

Feature Request: Enable Sending AdaptiveCards to Webex for Enhanced Alert Notifications

Open
#3,925 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind/enhancement
Dominant language
Go
Stars
8.6k
Forks
2.5k
Avg merge
2d 6h
Merged PRs (30d)
61

Description

What did you do?

I noticed that Alertmanager currently does not support sending AdaptiveCards to Webex, which can be very useful for enhancing the clarity and interactivity of alert notifications. Webex supports AdaptiveCards via its API with a specific format, which allows for richer message content including buttons and dropdown menus.

What did you expect to see?

I expected to see an option to configure Alertmanager to send alert notifications as AdaptiveCards to Webex, leveraging the Webex API format to enhance the alert details and provide interactive elements directly in the notification.

What did you see instead? Under which circumstances?

Currently, Alertmanager can send basic messages to Webex, but it lacks support for the richer formatting and interactivity provided by AdaptiveCards.

Environment

  • System information:

    Linux 6.5.0-44-generic x86_64

  • Alertmanager version:

alertmanager, version 0.27.0 (branch: main, revision: cad5fa580108431e6ed209f2a23a373aa50c098f)
build date: 20240728-09:39:12
go version: go1.21.1
platform: linux/amd64
tags: netgo

Alertmanager configuration file:
global:
  webex_api_url: https://webexapis.com/v1/messages

route:
  receiver: 'webex'
  group_by: ['alertname', 'cluster']
  group_wait: 15s
  group_interval: 15s
  repeat_interval: 30s

receivers:
- name: 'webex'
  webex_configs:
  - send_resolved: true
    room_id: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    adaptive_card: |-
      {
        "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
        "version": "1.2",
        "type": "AdaptiveCard",
        "body": [
          {
            "type": "TextBlock",
            "text": "{{ .Status | title }}",
            "size": "large",
            "color": "light",
            "weight": "bolder"
          },
          {{- range $index, $alert := .Alerts -}}
          {{- range $alert.Annotations.SortedPairs }}
            {{- if eq .Name "message" -}}
              {
              "type": "TextBlock",
              "text": "{{- .Value -}}",
              "color": "Accent",
              "size": "Medium",
              "wrap": true
              },
            {{- end }}
          {{- end -}}
          {
            "type": "ColumnSet",
            "columns": [
              {
                "type": "Column",
                "width": "1px",
                "style": "{{- if eq $.Status "resolved" -}}good{{- else -}}warning{{- end -}}",
                "items": []
              },
              {
                "type": "Column",
                "width": "stretch",
                "items": [
                  {
                    "type": "ColumnSet",
                    "columns": [
                      {
                        "type": "Column",
                        "width": "120px",
                        "items": [
                          {
                            "type": "TextBlock",
                            "text": "Alertname:",
                            "weight": "Bolder",
                            "color": "Light"
                          },
                          {
                            "type": "TextBlock",
                            "text": "Namespace:",
                            "color": "Light",
                            "spacing": "Small",
                            "weight": "Bolder"
                          },
                          {
                            "type": "TextBlock",
                            "text": "Job:",
                            "color": "Light",
                            "spacing": "Small",
                            "weight": "Bolder"
                          }
                        ]
                      },
                      {
                        "type": "Column",
                        "width": "stretch",
                        "items": [
                          {
                            "type": "TextBlock",
                            "text": "{{- $.GroupLabels.alertname -}}",
                            "color": "Light",
                            "spacing": "Small"
                          },
                          {
                            "type": "TextBlock",
                            "text": "{{- $.GroupLabels.namespace -}}",
                            "color": "Light",
                            "spacing": "Small"
                          },
                          {
                            "type": "TextBlock",
                            "text": "{{- $.GroupLabels.job -}}",
                            "color": "Light",
                            "spacing": "Small"
                          }
                        ]
                      }
                    ]
                  },
                  {
                    "type": "ColumnSet",
                    "columns": [
                      {
                        "type": "Column",
                        "width": "stretch",
                        "items": [
                          {
                            "type": "ColumnSet"
                          },
                          {
                            "type": "TextBlock",
                            "text": "Labels:",
                            "separator": true,
                            "color": "Light",
                            "weight": "Bolder"
                          },

                          {
                            "type": "ColumnSet",
                            "columns": [
                              {
                                "type": "Column",
                                "width": "20px"
                              },
                              {
                                "type": "Column",
                                "width": "stretch",
                                "items": [
                                  {
                                    "type": "FactSet",
                                    "facts": [
                                      {{- range $i, $e := $alert.Labels.SortedPairs }}{{- if $i -}},{{- end -}}
                                      {
                                        "title": "{{ $e.Name }}",
                                        "value": "{{ $e.Value }}"
                                      }
                                      {{- end }}
                                    ]
                                  }
                                ]
                              }
                            ]
                          },
                          {
                            "type": "ColumnSet",
                            "columns": [
                              {
                                "type": "Column",
                                "width": "stretch",
                                "items": [
                                  {
                                    "type": "TextBlock",
                                    "text": "Annotations:",
                                    "weight": "Bolder",
                                    "color": "Light",
                                    "separator": true
                                  },
                                  {
                                    "type": "ColumnSet",
                                    "columns": [
                                      {
                                        "type": "Column",
                                        "width": "20px"
                                      },
                                      {
                                        "type": "Column",
                                        "width": "stretch",
                                        "items": [
                                          {
                                            "type": "FactSet",
                                            "facts": [
                                              {{- range $i, $e := $alert.Annotations.SortedPairs  }}
                                                {{- if eq $e.Name "message" -}}
                                                  {{- if $i -}},{{- end -}}
                                                  {
                                                  "title": "{{ $e.Name }}",
                                                  "value": "{{ $e.Value }}"
                                                  }
                                                {{- end -}}
                                              {{- end }}
                                            ]
                                          }
                                        ]
                                      }
                                    ]
                                  }
                                ]
                              }
                            ]
                          }
                        ]
                      }
                    ]
                  }
                ]
              }
            ]
          },
          {
            "type": "ActionSet",
            "actions": [
              {
                "type": "Action.OpenUrl",
                "title": "📊 Alert",
                "url": "{{- $.ExternalURL -}}"
              },
              {
                "type": "Action.OpenUrl",
                "title": "🔕 Silence",
                "url": "{{- $.ExternalURL -}}/#/silences/new?filter=%7B{{- range $i, $e := $alert.Labels.SortedPairs -}}{{- if $i -}}%2C%20{{- end -}}{{- printf "%s%c3D%c22%s%c22" $e.Name 37 37 $e.Value 37 -}}{{- end -}}%7D"
              }
            ]
          }
          {{- end }}
        ]
      }
    http_config: 
      authorization:
        type: Bearer
        credentials: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
Send alert file (./alert.sh)
#!/bin/bash

name=$RANDOM
url='http://localhost:9093/api/v2/alerts'

echo "firing up alert $name" 

curl -XPOST $url -H "accept: application/json" -H "Content-Type: application/json" -d "[{ 
	\"status\": \"firing\",
	\"labels\": {
		\"alertname\": \"$name\",
		\"service\": \"my-service\",
		\"severity\":\"warning\",
		\"instance\": \"$name.example.net\"
	},
	\"annotations\": {
		\"summary\": \"High latency is high!\"
	},
	\"generatorURL\": \"http://prometheus.int.example.net/<generating_expression>\"
}]"

echo ""

echo "press enter to resolve alert"
read

echo "sending resolve"
curl -XPOST $url -H "accept: application/json" -H "Content-Type: application/json" -d "[{ 
	\"status\": \"resolved\",
	\"labels\": {
		\"alertname\": \"$name\",
		\"service\": \"my-service\",
		\"severity\":\"warning\",
		\"instance\": \"$name.example.net\"
	},
	\"annotations\": {
		\"summary\": \"High latency is high!\"
	},
	\"generatorURL\": \"http://prometheus.int.example.net/<generating_expression>\"
}]"

echo ""

Suggested Implementation:

I have already made the modifications and will be proposing a Pull Request (PR) soon.

The AdaptiveCard format that Webex supports is as follows:

{
  "attachments": {
    "contentType": "application/vnd.microsoft.card.adaptive",
    "content": {
      ...
    }
  }
}

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Alertmanager's existing Webex notification path and the webex_configs configuration shown in the issue. Define how an AdaptiveCard payload is configured and sent through the Webex API, then verify that both firing and resolved alerts support the requested richer notification format.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.