Monitoring metrics with Grafana in Cloudera Edge Management

Enabling Prometheus metrics in in Cloudera Edge Management

Ensure that the following metrics exporting property is enabled in in Cloudera Edge Management in the efm.properties file:

management.metrics.export.prometheus.enabled=true

efm.dashboard.base-url=http://grafana.example.com:3000
efm.dashboard.url.agentclass=/d/efm-agent-class/?var-agentClass={agentClass}

The base-url must reflect the location where you host Grafana. For details, see the Setting up Grafana section.

The dashboard URLs must point to the locations where you have set up agent and agent class specific URLs (see below).

Setting up Prometheus

1. Install Prometheus on a host that has network connectivity to in Cloudera Edge Management. For instructions about how to install Prometheus, see the Prometheus website.
2. Configure your prometheus.yml file to scrape the in Cloudera Edge Management instance. For example:

   # Global config

   global:
   scrape_interval: 1m
   evaluation_interval: 1m

   # The following is a scrape configuration for in Cloudera Edge Management. Add this to any other scrape configurations you desire. In this example, it is Prometheus.

   scrape_configs:
   - job_name: 'cem-efm'
   metrics_path: '/efm/actuator/prometheus'
     scrape_interval: 15s
     static_configs:
     - targets: ['efm.example.com:10090']

   For additional scrape configuration properties, such as TLS settings, see the Prometheus configuration guide.
3. Verify Prometheus configuration.

Setting up Grafana

1. Install Grafana on a host that has network connectivity to Prometheus. For instructions about how to install Grafana, see the Grafana website.
2. Configure Grafana to use Prometheus as a datasource. This can be done through the Grafana UI or through a datasources.yml provisioning file in conf/provisioning/datasources. For example:

   # config file version
   apiVersion: 1

   # List of datasources to insert or update depending on what is available in the database:

   apiVersion: 1
   
   datasources:
   - name: EFM Prometheus
   type: prometheus
   access: proxy
   orgId: 1
     url: http://prometheus.example.com:9090
   password:
   user:
   database:
   basicAuth: false
   basicAuthUser:
   basicAuthPassword:
   withCredentials: false
   isDefault: true
   jsonData:
     graphiteVersion: "1.1"
     tlsAuth: false
     tlsAuthWithCACert: false
   secureJsonData:
     tlsCACert: "..."
     tlsClientCert: "..."
     tlsClientKey: "..."
   version: 1
     editable: true

   For more information about configuring a Prometheus datasource in Grafana, see the Prometheus data source documentation.
3. Download the in Cloudera Edge Management Grafana dashboard templates.

   You can download the Grafana dashboard templates from Cloudera GitHub.

4. Import each JSON dashboard through the Grafana UI. Follow the Import dashboards instructions provided in the Grafana documentation.
   Alternatively, you can create a dashboards provider config file in conf/provisioning/dashboards to import dashboards.

   apiVersion: 1
   
   providers:
   - name: 'efm-prometheus'
     orgId: 1
     folder: 'EFM Prometheus'
     type: file
     disableDeletion: false
     editable: true
     options:
     path: /tmp/dashboards

   Update the path property as per your requirements and also create the directory in the local filesystem. Place the downloaded dashboard definitions in the created directory and start Grafana.

   in Cloudera Edge Management dashboards should now be available in the Grafana UI.

Navigating to Grafana dashboard in in Cloudera Edge Management

To navigate to class specific Grafana dashboards, select a class in the Dashboard screen. The Metrics tab appears for the class. Select the View Grafana Dashboard link in the Information section.

For more information about the integration with Prometheus and Grafana, check out the video on the Cloudera Edge Management YouTube playlist: