Monitoring metrics in CEM with Grafana

Cloudera Edge Management (CEM) can export time series metrics to several metric storage providers. The recommended metrics store service is Prometheus. Prometheus integrates with Grafana for time series metric visualization. With Prometheus and Grafana, you can store and visualize metrics for CEM.

You need to perform the following tasks before you start visualizing CEM metrics with Prometheus and Grafana.

Enabling Prometheus metrics in CEM

Ensure that the following metrics exporting property is enabled in CEM in the efm.properties file:

management.metrics.export.prometheus.enabled=true
You need to customize the following efm.dashboard.* properties:
efm.dashboard.base-url=http://grafana.example.com:3000
efm.dashboard.url.agentclass=/d/efm-agent-class/?var-agentClass={agentClass}

The base-url must reflect the location where you host Grafana. For details, see the Setting up Grafana section.

The dashboard URLs must point to the locations where you have set up agent and agent class specific URLs (see below).

Setting up Prometheus

  1. Install Prometheus on a host that has network connectivity to CEM. For instructions about how to install Prometheus, see https://prometheus.io.
  2. Configure your prometheus.yml file to scrape the CEM instance. For example:

    # Global config

    global:
    scrape_interval: 1m
    evaluation_interval: 1m
    # The following is a scrape configuration for CEM. Add this to any other scrape configurations you desire. In this example, it is Prometheus.
    scrape_configs:
    - job_name: 'cem-efm'
    metrics_path: '/efm/actuator/prometheus'
      scrape_interval: 15s
      static_configs:
      - targets: ['efm.example.com:10090']
    For additional scrape configuration properties, such as TLS settings, see the Prometheus configuration guide: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config.
  3. Verify Prometheus configuration.

Setting up Grafana

  1. Install Grafana on a host that has network connectivity to Prometheus. For instructions about how to install Grafana, see https://grafana.com.
  2. Configure Grafana to use Prometheus as a datasource. This can be done through the Grafana UI or through a datasources.yml provisioning file in conf/provisioning/datasources. For example:
    # config file version
    apiVersion: 1
    # List of datasources to insert or update depending on what is available in the database:
    apiVersion: 1
    
    datasources:
    - name: EFM Prometheus
    type: prometheus
    access: proxy
    orgId: 1
      url: http://prometheus.example.com:9090
    password:
    user:
    database:
    basicAuth: false
    basicAuthUser:
    basicAuthPassword:
    withCredentials: false
    isDefault: true
    jsonData:
      graphiteVersion: "1.1"
      tlsAuth: false
      tlsAuthWithCACert: false
    secureJsonData:
      tlsCACert: "..."
      tlsClientCert: "..."
      tlsClientKey: "..."
    version: 1
      editable: true
    For more information about configuring a Prometheus datasource in Grafana, see https://grafana.com/docs/features/datasources/prometheus/.
  3. Download the CEM Grafana dashboard templates.

    You can download the Grafana dashboard templates from https://github.com/cloudera/cloudera-edge-management-grafana-boards.

  4. Import each JSON dashboard through the Grafana UI. Follow the instructions provided in the following Grafana document: https://grafana.com/docs/reference/export_import/#importing-a-dashboard.
    Alternatively, you can create a dashboards provider config file in conf/provisioning/dashboards to import dashboards.
    apiVersion: 1
    
    providers:
    - name: 'efm-prometheus'
      orgId: 1
      folder: 'EFM Prometheus'
      type: file
      disableDeletion: false
      editable: true
      options:
      path: /tmp/dashboards
    Update the path property as per your requirements and also create the directory in the local filesystem. Place the downloaded dashboard definitions in the created directory and start Grafana.

    CEM dashboards should now be available in the Grafana UI.

Navigating to Grafana dashboard in CEM

To navigate to class specific Grafana dashboards, select a class in the Dashboard screen. The Metrics tab appears for the class. Select the View Grafana Dashboard link in the Information section.