This is the documentation for Cloudera Manager 5.1.x. Documentation for other versions is available at Cloudera Documentation.

Monitoring Resource Management

Cloudera Manager provides several features to assist you with allocating cluster resources to services. Initially Cloudera Manager introduced the ability to partition resources across HBase, HDFS, Impala, MapReduce, and YARN services by setting configuration properties that were enforced by Linux control groups (Linux cgroups). With Cloudera Manager 5, the ability to statically allocate resources using cgroups is configurable through a single static service pool user interface.

Monitoring Static Service Pools

Static service pools isolate the services in your cluster from one another, so that load on one service has a bounded impact on other services. Services are allocated a static percentage of total resources—CPU, memory, and I/O weight—which are not shared with other services. When you configure static service pools, Cloudera Manager computes recommended memory, CPU, and I/O configurations for the worker roles of the services that correspond to the percentage assigned to each service. Static service pools are implemented per role group within a cluster, using Linux control groups (cgroups) and cooperative memory limits (for example, Java maximum heap sizes). Static service pools can be used to control access to resources by HBase, HDFS, Impala, MapReduce, Solr, Spark, YARN, and add-on services. Static service pools are not enabled by default.

Viewing Static Service Pools

Select Clusters > ClusterName > Static Service Pools. If the cluster has a YARN service, the Static Service Pools Status tab displays and shows whether resource management is enabled for the cluster, and the currently configured service pools.

Static Service Pool Status

The Status tab of the Static Service Pools page contains a list of current services that can or have been allocated resources and a set of resource usage charts for the cluster.

Click Historical Data to display detailed resource usage charts for each service.

Click a duration link at the top right of the charts to change the time period for which the resource usage displays.

Monitoring Dynamic Resource Pools

A dynamic resource pool is a named configuration of resources and a policy for scheduling the resources among YARN applications or Impala queries running in the pool. Dynamic resource pools allow you to schedule and allocate resources to YARN applications and Impala queries based on the user's access to specific pools and the resources available to those pools. If a pool's allocation is not in use it can be given to other pools. Otherwise, a pool receives a share of resources in accordance with the pool’s weight. Dynamic resource pools have ACLs that restrict who can submit work to and administer them.

Viewing Dynamic Resource Pools

  1. Select Clusters > ClusterName > Dynamic Resource Pools. If the cluster has a YARN service, the Dynamic Resource Pools Status tab displays. If the cluster has only an Impala service, the Configuration tab displays.

Dynamic Resource Pools Status

For the YARN Independent RM and YARN and Impala Integrated RM resource management scenarios described in Managing Resources, the Dynamic Resource Pools Status tab displays a summary of YARN scheduler status, a list of current allocations for the pools that can or have been allocated resources, and a set of pool resource usage charts for the cluster. If the Impala Independent RM scenario is in effect, there is no Status tab.

Click a duration link at the top right of the charts to change the time period for which the resource usage displays.

  • Status - a summary of the virtual CPU cores and memory that can be allocated by the YARN scheduler.
  • Pools Status - a list of pools that have been explicitly configured and pools created by YARN, properties of the pools, and an action menu.
    • Allocated Memory - The memory assigned to the pool that is currently allocated to applications and queries.
    • Allocated VCores - The number of virtual CPU cores assigned to the pool that are currently allocated to applications and queries.
    • Allocated Containers - The number of YARN containers assigned to the pool whose resources have been allocated.
    • Pending Containers - The number of YARN containers assigned to the pool whose resources are pending.
    • Click and select
      • YARN Applications to display the YARN Applications page and list the applications that are running or have run in that pool.
      • Impala Queries to display the Impala Queries page and list the queries that are running or have run in that pool.
Page generated September 3, 2015.