Microsoft Teams®: Send DKP Insights Alert Notifications to a Channel
This page contains information on how to set up a configuration for Alertmanager to send alert notifications through Microsoft Teams®. See DKP Insights Alert Notifications With Alertmanager for more information about this function.
Prerequisites
Kube Prometheus Stack installed on the Management cluster (included in the default configuration)
CLI Configuration
Prepare your Environment
Set your environment variable to the
kommander
workspace namespace:CODEexport WORKSPACE_NAMESPACE=kommander
Set the Microsoft Teams® Webhook variable to the URL you obtained from Microsoft for this purpose:
The webhook format is similar tohttps://xxxxx.webhook.office.com/xxxxxxxxx
.CODEexport TEAMS_WEBHOOK=<endpoint_URL>
Enable DKP Insights to Send Notifications with Alertmanager
Install an extension for Kube Prometheus Stack that adds compatibility with Microsoft Teams®. Then, Create an AlertmanagerConfig
object and apply it on the kommander
workspace namespace.
Add the following repository to enable Microsoft Teams® configuration:
CODEhelm repo add prometheus-msteams https://prometheus-msteams.github.io/prometheus-msteams/
Create a custom configuration of Kube Prometheus Stack, and name it
teams-proxy-config.yaml
:
Replace<teams_webhook_URL>
with the webhook you obtained from Microsoft Teams®. The format is similar tohttps://xxxxx.webhook.office.com/xxxxxxxxx
.CODEreplicaCount: 1 image: repository: quay.io/prometheusmsteams/prometheus-msteams tag: v1.5.1 connectors: - alertmanager: <teams_webhook_URL> container: additionalArgs: - -debug metrics: serviceMonitor: enabled: true additionalLabels: release: kube-prometheus-stack-prometheus scrapeInterval: 30s
Create a custom display format for your message in Microsoft Teams® message, and name the file
custom-card.tmpl
:CODE{{ define "teams.card" }} { "@type": "MessageCard", "@context": "http://schema.org/extensions", "themeColor": "{{- if eq .Status "resolved" -}}2DC72D {{- else if eq .Status "Firing" -}} {{- if eq .CommonLabels.severity "Critical" -}}8C1A1A {{- else if eq .CommonLabels.severity "Warning" -}}FFA500 {{- else -}}808080{{- end -}} {{- else -}}808080{{- end -}}", "summary": "{{- if eq .CommonAnnotations.description "" -}} {{- if eq .CommonLabels.insightClass "" -}} {{- if eq .CommonLabels.alertname "" -}} Prometheus Alert {{- else -}} {{- .CommonLabels.alertname -}} {{- end -}} {{- else -}} {{- .CommonLabels.insightClass -}} {{- end -}} {{- else -}} {{- .CommonAnnotations.description -}} {{- end -}}", "title": "{{ .Status | toUpper -}}{{ if eq .Status "firing" }}: {{ .Alerts.Firing | len }} {{- end}} Insights Alert{{ if gt (len .Alerts.Firing) 1 }}s{{ end }} ({{ .CommonLabels.insightClass }})", "sections": [ {{$externalUrl := (index .Alerts 0).Annotations.detailsURL }} { "activityTitle": "[{{ (index .Alerts 0).Annotations.description }}]({{ $externalUrl }})", "facts": [ {{- if (index .Alerts 0).Labels.namespace }} { "name": "Namespace:", "value": "{{ (index .Alerts 0).Labels.namespace }}" }, {{- end }} {{- if (index .Alerts 0).Labels.severity }} { "name": "Severity:", "value": "{{ (index .Alerts 0).Labels.severity }}" }, {{- end }} {{- if (index .Alerts 0).Labels.cluster }} { "name": "Cluster:", "value": "{{ (index .Alerts 0).Labels.cluster }}" }, {{- end }} {{- if (index .Alerts 0).Annotations.categories }} { "name": "Categories:", "value": "{{ (index .Alerts 0).Annotations.categories }}" } {{- end }} ], "markdown": true } ] } {{ end }}
Upgrade the Helm values to apply the above configuration:
CODEhelm upgrade --install prometheus-msteams \ --namespace ${WORKSPACE_NAMESPACE} -f teams-proxy-config.yaml \ --set-file customCardTemplate=custom-card.tmpl \ prometheus-msteams/prometheus-msteams
Create the
AlertmanagerConfig
YAML file and name italertmanager-teams-config.yaml
.
This example allows Alertmanager to send notifications for all Critical alerts of all alert types that occur in any workspace to Microsoft Teams®.CODEapiVersion: monitoring.coreos.com/v1alpha1 kind: AlertmanagerConfig metadata: name: alertmanager-teams-config.yaml namespace: kommander spec: route: groupBy: ['source', 'insightClass', 'severity', 'cluster'] groupWait: 3m groupInterval: 15m repeatInterval: 1h receiver: 'default' routes: - receiver: 'teams' matchers: - name: source value: Insights matchType: = - name: severity value: Critical matchType: = continue: true receivers: - name: 'default' - name: 'teams' webhookConfigs: - url: 'http://prometheus-msteams:2000/alertmanager'
Apply the
AlertmanagerConfig
file:CODEkubectl -n ${WORKSPACE_NAMESPACE} apply -f alertmanager-teams-config.yaml