Monitoring metrics in CEM with Grafana
Cloudera Edge Management (CEM) can export time series metrics to several metric storage providers. The recommended metrics store service is Prometheus. Prometheus integrates with Grafana for time series metric visualization. With Prometheus and Grafana, you can store and visualize metrics for CEM.
You need to perform the following tasks before you start visualizing CEM metrics with Prometheus and Grafana.
Enabling Prometheus metrics in CEM
Ensure that the following metrics exporting property is enabled in CEM in the
The base-url must reflect the location where you host Grafana. For details, see the Setting up Grafana section.
The dashboard URLs must point to the locations where you have set up agent and agent class specific URLs (see below).
Setting up Prometheus
- Install Prometheus on a host that has network connectivity to CEM. For instructions about how to install Prometheus, see https://prometheus.io.
- Configure your
prometheus.ymlfile to scrape the CEM instance. For example:
# Global config
# The following is a scrape configuration for CEM. Add this to any other scrape configurations you desire. In this example, it is Prometheus.
global: scrape_interval: 1m evaluation_interval: 1m
For additional scrape configuration properties, such as TLS settings, see the Prometheus configuration guide: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config.
scrape_configs: - job_name: 'cem-efm' metrics_path: '/efm/actuator/prometheus' scrape_interval: 15s static_configs: - targets: ['efm.example.com:10090']
- Verify Prometheus configuration.
Setting up Grafana
- Install Grafana on a host that has network connectivity to Prometheus. For instructions about how to install Grafana, see https://grafana.com.
- Configure Grafana to use Prometheus as a datasource. This can be done through the
Grafana UI or through a
datasources.ymlprovisioning file in
conf/provisioning/datasources. For example:
# List of datasources to insert or update depending on what is available in the database:
# config file version apiVersion: 1
For more information about configuring a Prometheus datasource in Grafana, see https://grafana.com/docs/features/datasources/prometheus/.
apiVersion: 1 datasources: - name: EFM Prometheus type: prometheus access: proxy orgId: 1 url: http://prometheus.example.com:9090 password: user: database: basicAuth: false basicAuthUser: basicAuthPassword: withCredentials: false isDefault: true jsonData: graphiteVersion: "1.1" tlsAuth: false tlsAuthWithCACert: false secureJsonData: tlsCACert: "..." tlsClientCert: "..." tlsClientKey: "..." version: 1 editable: true
- Download the CEM Grafana dashboard templates.
You can download the Grafana dashboard templates from https://github.com/cloudera/cloudera-edge-management-grafana-boards.
- Import each JSON dashboard through the Grafana UI. Follow the instructions provided in
the following Grafana document: https://grafana.com/docs/reference/export_import/#importing-a-dashboard.Alternatively, you can create a dashboards provider config file in
conf/provisioning/dashboardsto import dashboards.
Update the path property as per your requirements and also create the directory in the local filesystem. Place the downloaded dashboard definitions in the created directory and start Grafana.
apiVersion: 1 providers: - name: 'efm-prometheus' orgId: 1 folder: 'EFM Prometheus' type: file disableDeletion: false editable: true options: path: /tmp/dashboards
CEM dashboards should now be available in the Grafana UI.
Navigating to Grafana dashboard in CEM
To navigate to class specific Grafana dashboards, select a class in the Dashboard screen. The Metrics tab appears for the class. Select the View Grafana Dashboard link in the Information section.
For more information about the integration with Prometheus and Grafana, check out the video on the Cloudera Edge Management YouTube playlist: