Cloudera Docs

Enable the Cluster Utilization Report

You must configure several parameters to enable the Cluster Utilization Report.

Minimum Required Role: Configurator (also provided by Cluster Administrator, Limited Cluster Administrator , and Full Administrator)

By default, the Cluster Utilization Report displays aggregated CPU and memory utilization for an entire cluster and for YARN and Impala utilization. You can also view this utilization by tenants, which include Linux users and Dynamic Resource Pools. To see utilization for a tenant, you must configure the tenant and define resource limits for it.

Enable YARN utilization metrics collection:

  1. In Cloudera Manager, select the YARN service.
  2. Click the Configuration tab.
  3. Search for container.
  4. Find the Enable Container Usage Metrics Collection property and select it.
  5. Find the Container Usage MapReduce Job User property.
  6. Enter a username for the MapReduce job that collects the metrics.
    The username you enter must be a Linux user on all the cluster hosts. If you are using an Active Directory KDC, the username must also exist in Active Directory. For secure clusters, the user must not be banned or below the minimum user ID. You can view the list of banned users (banned.users) and the minimum user ID (min.user.id) by under YARN Configuration in Cloudera Manager.

    The user that is configured with the Container Usage MapReduce Job User property in the YARN service requires permissions to read the subdirectories of the HDFS directory specified with the Cloudera Manager Container Usage Metrics Directory property. The default umask of 022 allows any user to read from that directory. However, if a more strict umask (for example, 027) is used, then those directories are not readable by any user. In that case the user specified with the Container Usage MapReduce Job User property should be added to the same group that owns the subdirectories.

    For example, if the /tmp/cmYarnContainerMetrics/20161010 subdirectory is owned by user and group yarn:hadoop, the user specified in Container Usage MapReduce Job User should be added to the hadoop group.

    The directories you specify with the Cloudera Manager Container Usage Metrics Directory and Container Usage Output Directory properties should not be located in encryption zones.

  7. Optionally, enter the resource pool in which the container usage collection MapReduce job runs in the Container Usage MapReduce Job Pool property.

    Cloudera recommends that you dedicate a resource pool for running this MapReduce job.

    If you specify a custom resource pool, ensure that the placement rules for the cluster allow for it. The first rule must be for resource pools to be specified at run time with the Create pool if it does not exist option selected. Alternatively, ensure that the pool you specify already exists. If the placement rule is not properly configured or the resource pool does not already exist, the job may run in a different pool.

  8. Click Save Changes.
  9. Click the Actions button.
  10. Select Crate CM Container Usage Metrics Dir.
  11. Restart the YARN service.

Enable Impala utilization collection:

  1. In Cloudera Manager, select the Impala service.
  2. Click the Configuration tab.
  3. Search for admission control.
  4. Find the Enable Impala Admission Control and the Enable Dynamic Resource Pools properties and clear both of them.
  5. Click Save Changes.
  6. Restart the Impala service.