Slack®: Send DKP Insights Alert Notifications to a Channel

This page contains information on how to set up a configuration for Alertmanager to send alert notifications through Slack®*. See DKP Insights Alert Notifications With Alertmanager for more information about this function.

Prerequisites

• Kube Prometheus Stack installed on the Management cluster (included in the default configuration)

• A Slack® Incoming Webhook created by a Slack® workspace admin

• DKP Insights installed

CLI Configuration

Prepare your Environment

1. Set your environment variable to the kommander workspace namespace:

   CODE

   export WORKSPACE_NAMESPACE=kommander

2. Set the Slack® Webhook variable to the URL you obtained from Slack® for this purpose:
   The webhook format is similar to https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX.

   CODE

   export SLACK_WEBHOOK=<endpoint_URL>

Enable DKP Insights to Send Notifications with Alertmanager

Create an AlertmanagerConfig object and apply it on the kommander workspace namespace.

1. Create a secret for the Alertmanager-Slack integration:

   CODE

   kubectl create secret generic slack-webhook -n ${WORKSPACE_NAMESPACE} \
     --from-literal=slack-webhook-url=${SLACK_WEBHOOK} \
     --dry-run=client --save-config -o yaml | kubectl apply -f -

2. Create the AlertmanagerConfig YAML file and name it alertmanager-slack-config.yaml.
   This example allows Alertmanager to send notifications for all Critical Insights of all alert types that occur in any workspace to Slack®.
   Replace <#target_slack_channel> with the name of the Slack® channel where you want to receive the notifications.

   CODE

   apiVersion: monitoring.coreos.com/v1alpha1
   kind: AlertmanagerConfig
   metadata:
     name: slack-config
     namespace: kommander
   spec:
     route:
       groupBy: ['source', 'insightClass', 'severity', 'cluster']
       groupWait: 3m
       groupInterval: 15m
       repeatInterval: 1h
       receiver: 'slack'
       routes:
         - receiver: 'slack'
           matchers:
             - name: source
               value: Insights
               matchType: =
             - name: severity
               value: Critical
               matchType: =
           continue: true
     receivers:
       - name: 'slack'
         slackConfigs:
           - apiURL:
               name: slack-webhook
               key: slack-webhook-url
             channel: '#<target_slack_channel>'
             username: Insights Slack Notifier
             iconURL: https://avatars3.githubusercontent.com/u/3380462
             title: |-
               {{ .Status | toUpper -}}{{ if eq .Status "firing" }}: {{ .Alerts.Firing | len }} {{- end}} Insights Alert{{ if gt (len .Alerts.Firing) 1 }}s{{ end }} ({{ .CommonLabels.insightClass }})
             titleLink: 'https://{{ (index .Alerts 0).Annotations.detailsURL }}'
             text: |-
               {{- if (index .Alerts 0).Labels.namespace }}
                 {{- "\n" -}}
                 *Namespace:* `{{ (index .Alerts 0).Labels.namespace }}`
               {{- end }}
               {{- if (index .Alerts 0).Labels.severity }}
                 {{- "\n" -}}
                 *Severity:* `{{ (index .Alerts 0).Labels.severity }}`
               {{- end }}
               {{- if (index .Alerts 0).Labels.cluster }}
                 {{- "\n" -}}
                 *Cluster:* `{{ (index .Alerts 0).Labels.cluster }}`
               {{- end }}
               {{- if (index .Alerts 0).Annotations.description }}
                 {{- "\n" -}}
                 *Description:* {{ (index .Alerts 0).Annotations.description }}
               {{- end }}
               {{- if (index .Alerts 0).Annotations.categories }}
                 {{- "\n" -}}
                 *Categories:* {{ (index .Alerts 0).Annotations.categories }}
               {{- end }}
             actions:
               - text: 'Go to Insight :mag:'
                 type: button
                 url: 'https://{{ (index .Alerts 0).Annotations.detailsURL }}'

3. Apply the AlertmanagerConfig file:

   CODE

   kubectl -n ${WORKSPACE_NAMESPACE} apply -f alertmanager-slack-config.yaml

Next Step:

Verify That Alertmanager Sends Notifications

*Slack® is a trademark and service mark of Slack Technologies, Inc., registered in the U.S. and in other countries.