Monitoring metrics in CEM with Grafana

Cloudera Edge Management (CEM) can export time series metrics to several metric storage providers. The recommended metrics store service is Prometheus. Prometheus integrates with Grafana for time series metric visualization. With Prometheus and Grafana, you can store and visualize metrics for CEM.

You need to perform the following tasks before you start visualizing CEM metrics with Prometheus and Grafana.

Enabling Prometheus metrics in CEM

Ensure that the following metrics exporting property is enabled in CEM in the efm.properties file:

management.metrics.export.prometheus.enabled=true

You need to customize the following efm.dashboard.* properties:

efm.dashboard.base-url=http://grafana.example.com:3000
efm.dashboard.url.agentclass=/d/efm-agent-class/?var-agentClass={agentClass}
efm.dashboard.url.agent=/d/efm-agent/?var-agentId={agentId}

The base-url must reflect the location where you host Grafana. For details, see the Setting up Grafana section.

The dashboard URLs must point to the locations where you have set up agent and agent class specific URLs (see below).

Setting up Prometheus

Install Prometheus on a host that has network connectivity to CEM. For instructions to install Prometheus, see https://prometheus.io.
Configure your prometheus.yml file to scrape the CEM instance. For example:
# Global config
```
global:
scrape_interval: 1m
evaluation_interval: 1m
```
# The following is a scrape configuration for CEM. Add this to any other scrape configurations you desire. In this example, it is Prometheus.
```
scrape_configs:
- job_name: 'cem-efm'
metrics_path: '/efm/actuator/prometheus'
scrape_interval: 15s
```
```
static_configs:
- targets: ['efm.example.com:10080']
```
For additional scrape configuration properties, such as TLS settings, see the Prometheus configuration guide: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config.
Verify Prometheus configuration.

Setting up Grafana

Install Grafana on a host that has network connectivity to Prometheus. For instructions to install Grafana, see https://grafana.com.

Configure Grafana to use Prometheus as a datasource. This can be done through the Grafana UI or through a datasources.yml provisioning file. For example:

# config file version
apiVersion: 1

# List of datasources that must be deleted from the database:

deleteDatasources:
- name: CEM EFM Prometheus orgId: 1

# List of datasources to insert or update depending what is available in the database:

datasources:
- name: CEM EFM Prometheus
type: prometheus
access: proxy
orgId: 1
url: http://prometheus:9090
password:
user:
database:
basicAuth: false
basicAuthUser:
basicAuthPassword:
withCredentials: false
isDefault: true
jsonData:
  graphiteVersion: "1.1"
  tlsAuth: false
  tlsAuthWithCACert: false
secureJsonData:
  tlsCACert: "..."
  tlsClientCert: "..."
  tlsClientKey: "..."
version: 1
editable: true

For more information for configuring a Prometheus datasource in Grafana, see https://grafana.com/docs/features/datasources/prometheus/.

Download the CEM Grafana dashboard templates.
You can download the Grafana dashboard templates included in source files from https://www.cloudera.com/downloads/cdf.html.
Import each JSON dashboard. Follow the instructions provided in the following Grafana document: https://grafana.com/docs/reference/export_import/#importing-a-dashboard.
CEM dashboards should now be available in the Grafana UI.

Navigating to Grafana dashboard in CEM

To navigate to class specific Grafana dashboards, select a class in the Deployment screen. In the class page, click the About Class link at the top-right corner. Select the View Grafana Dashboard link in the About This Class window.

To navigate to agent specific Grafana dashboards, select an event source in the Events screen. In the event page, click the About Agent link at the top-right corner. Select the View Grafana Dashboard link in the About This Agent window.