Meeting prerequisites to set up alerts

Before you can set up an alert triggered by a dashboard event, you must configure an SMTP (Simple Mail Transfer Protocol) server, an alert recipient, and a notification policy.

  1. Configure grafana using kubectl.
    kubectl edit configmap grafana -n istio-system
    For more information, see "Granting remote access to Kubernetes".
  2. Update the grafana.ini to configure SMTP, setting user to the altert account email id and the password to the app password generated in your gmail account.
    ...
    [smtp]
    enabled = true
    host = smtp.gmail.com:465
    user = youremail@gmail.com
    password = substitute_your_password
    ;cert_file=
    ;key_file=
    skip_verify = true
    from_address = alertgrafan@gmail.com
    from_name = Grafana
    * EHLO identity in SMTP dialog (defaults to instance_name)
    ;ehlo_identity = dashboard.example.com
    ;startTLS_policy = NoStartTLS
    
  3. Refresh the grafana pod.
    kubectl get pod -A grep -i grafana
    kubectl delete pod grafana-9bcb5d5c-w8r6c -n istio-system
  4. Log into Grafana, and click Alerting > Contact points.
  5. In Contact point type, select an alert receiver type.
  6. In Addresses, enter an email address, and then in Message, compose an email message body.
    For example, enter an email message body test grafana email.
  7. Click Test to verify that a test alert arrives in your inbox.
  8. Scroll up and in Name, enter the contact point name.
  9. Log into Grafana, and click Alerting > Notification policies.
  10. Click New specific policy.
  11. In Contact Point, select the contact point to receive notification, and save the policy.