Cloudera Docs

Grafana in CDW overview

You connect to prebuilt dashboards to view metrics of CDW operations. Cloudera provides prebuilt Grafana dashboards for Hive, Impala, Hue, Druid, Kubernetes, and Istio dashboards of metrics data, charts, and other visuals.

Using Grafana, CDP metrics are centralized in a single spot, stored in the Prometheus database and monitored by Prometheus. Your workload databases are not involved in any way. You can immediately view pre-built dashboards described below. You can view dashboard metrics for different time periods by selecting the period of interest from the dropdown in the horizontal navigation:

You can also create your own dashboards. "Get started with Grafana and Prometheus" describes how to create dashboards of CDP metrics. Describing all the details of how to use Grafana is beyond the scope of this documentation. Grafana described in this documentation is not the enterprise version.

Hive dashboards

The Hive dashboards cover the following operations of the Hive SQL engine in CDW:
  • Auto-scaling
  • Hive metastore
  • HiveServer
  • The Hive service itself (Hive-Home)
  • LLAP
For example:

Impala dashboards

The Impala dashboards include the following operations of the Impala SQL engine in CDW:
  • Catalog server
  • Coordinator
  • Executor
  • Statestore
  • The Impala service itself

Kubernetes dashboards

You can get insight into the operations of your CDP clusters from Kubernetes dashboards. Kubernetes dashboards represent the following metrics:
  • CoreDNS: requests and duration of responses
  • App Metrics: number of Kubernetes pods, CPU usage relative to request or to limit, memory usage sliced and diced a number of ways
  • Object Metrics: Deployment memory and CPU usage, in total, and by node.

Istio dashboards

To work with Cloudera Support, you might use the Istio dashboards. Istio is an open platform that provides microservice security, connections, and monitoring. The Istio Mesh dashboards cover the following views of the service mesh network of microservices:
  • Istio Mesh summary: describes the network of microservices by HTTP/gRPC and TCP workloads in the Mesh.
  • Individual CDP services, such as HiveServer and Impala coordinator: Request and response metrics, such as latency, for each mesh service (HTTP/gRPC and TCP) and client and service workloads metrics.
  • Individual workloads using the services: Request and response metrics for each workload in the Mesh (HTTP/gRPC and TCP) and inbound/outbound workload services.
The Istio performance dashboard presents visualizations of the following metrics:
  • CDP Usage
  • Memory Usage
  • Data Rates
  • Bytes transferred per second

The Istio control plane dashboard includes memory, CPU, and disk resource usage.

For example:

The sidecar proxy metrics in Istio dashboards reveals the interceptions of network communication between microservices.

The Istio Wasm Plugin extends the Istio proxy capabilities.

Nodes dashboards

The Prometheus Node Exporter is used to gather detailed metrics for the AWS and Azure Virtual Machines that host an environment's Kubernetes cluster. These metrics cover both machine and OS level metrics such as CPU, memory, network, processes, time synchronization, disk, and file system. All metric names start with node_.

Three prebuild dashboards show metrics for AWS/Azure Virtual Machines that host the Kubernetes cluster:
  • Cluster Totals

    Shows CPU/Memory utilization and node counts (shared services/compute for the entire cluster.

  • Node Details

    Shows very detailed metrics for a single node at a time.

  • Node Trends

    Combines CPU, Memory, Disk, and Network metrics for all nodes for node-to-node comparisons.