Meeting prerequisites to set up alerts
Before you can set up an alert triggered by a dashboard event, you must configure an SMTP (Simple Mail Transfer Protocol) server, an alert recipient, and a notification policy.
-
Configure grafana using kubectl.
kubectl edit configmap grafana -n istio-system
For more information, see "Granting remote access to Kubernetes". -
Update the grafana.ini to configure SMTP, setting user to the altert account
email id and the password to the app password generated in your gmail
account.
... [smtp] enabled = true host = smtp.gmail.com:465 user = youremail@gmail.com password = substitute_your_password ;cert_file= ;key_file= skip_verify = true from_address = alertgrafan@gmail.com from_name = Grafana * EHLO identity in SMTP dialog (defaults to instance_name) ;ehlo_identity = dashboard.example.com ;startTLS_policy = NoStartTLS
-
Refresh the grafana pod.
kubectl get pod -A grep -i grafana kubectl delete pod grafana-9bcb5d5c-w8r6c -n istio-system
-
Log into Grafana, and click Alerting > Contact points.
- In Contact point type, select an alert receiver type.
-
In Addresses, enter an email address, and then in
Message, compose an email message body.
For example, enter an email message body test grafana email.
- Click Test to verify that a test alert arrives in your inbox.
- Scroll up and in Name, enter the contact point name.
-
Log into Grafana, and click Alerting > Notification policies.
-
Click New specific policy.
-
In Contact Point, select the contact point to receive notification, and save
the policy.