Cloudera Docs

YARN Metrics

Reference information for YARN Metrics

In addition to these base metrics, many aggregate metrics are available. If an entity type has parents defined, you can formulate all possible aggregate metrics using the formula base_metric_across_parents.

In addition, metrics for aggregate totals can be formed by adding the prefix total_ to the front of the metric name.

Use the type-ahead feature in the Cloudera Manager chart browser to find the exact aggregate metric name, in case the plural form does not end in "s".

For example, the following metric names may be valid for YARN:

  • alerts_rate_across_clusters
  • total_alerts_rate_across_clusters

Some metrics, such as alerts_rate, apply to nearly every metric context. Others only apply to a certain service or role.

alerts_rate

Description
The number of alerts.
Unit
events per second
Parents
cluster
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

apps_ingested_rate

Description
YARN applications ingested by the Service Monitor
Unit
Applications per second
Parents
cluster
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

events_critical_rate

Description
The number of critical events.
Unit
events per second
Parents
cluster
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

events_important_rate

Description
The number of important events.
Unit
events per second
Parents
cluster
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

events_informational_rate

Description
The number of informational events.
Unit
events per second
Parents
cluster
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

health_bad_rate

Description
Percentage of Time with Bad Health
Unit
seconds per second
Parents
cluster
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

health_concerning_rate

Description
Percentage of Time with Concerning Health
Unit
seconds per second
Parents
cluster
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

health_disabled_rate

Description
Percentage of Time with Disabled Health
Unit
seconds per second
Parents
cluster
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

health_good_rate

Description
Percentage of Time with Good Health
Unit
seconds per second
Parents
cluster
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

health_unknown_rate

Description
Percentage of Time with Unknown Health
Unit
seconds per second
Parents
cluster
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

yarn_application_adl_bytes_read_rate

Description
ADL bytes read. Called 'adl_bytes_read' in searches.
Unit
bytes per second
Parents
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_application_adl_bytes_written_rate

Description
ADL bytes written. Called 'adl_bytes_written' in searches.
Unit
bytes per second
Parents
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_application_application_duration_rate

Description
How long YARN took to execute this application. Called 'application_duration' in searches.
Unit
ms per second
Parents
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_application_cm_cpu_milliseconds_rate

Description
yarn.analysis.cm_cpu_milliseconds.description
Unit
ms per second
Parents
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_application_cpu_milliseconds_rate

Description
CPU time. Called 'cpu_milliseconds' in searches.
Unit
ms per second
Parents
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_application_file_bytes_read_rate

Description
File bytes read. Called 'file_bytes_read' in searches.
Unit
bytes per second
Parents
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_application_file_bytes_written_rate

Description
File bytes written. Called 'file_bytes_written' in searches.
Unit
bytes per second
Parents
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_application_hdfs_bytes_read_rate

Description
HDFS bytes read. Called 'hdfs_bytes_read' in searches.
Unit
bytes per second
Parents
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_application_hdfs_bytes_written_rate

Description
HDFS bytes written. Called 'hdfs_bytes_written' in searches.
Unit
bytes per second
Parents
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_application_maps_rate

Description
The number of Map tasks in this MapReduce job. Called 'maps_total' in searches.
Unit
items per second
Parents
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_application_mb_millis_maps_rate

Description
Map memory allocation. Called 'mb_millis_maps' in searches.
Unit
items per second
Parents
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_application_mb_millis_reduces_rate

Description
Reduce memory allocation. Called 'mb_millis_reduces' in searches.
Unit
items per second
Parents
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_application_reduces_rate

Description
The number of reduce tasks in this MapReduce job. Called 'reduces_total' in searches.
Unit
items per second
Parents
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_application_s3a_bytes_read_rate

Description
S3A bytes read. Called 's3a_bytes_read' in searches.
Unit
bytes per second
Parents
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_application_s3a_bytes_written_rate

Description
S3A bytes written. Called 's3a_bytes_written' in searches.
Unit
bytes per second
Parents
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_application_vcores_millis_maps_rate

Description
Map CPU allocation. Called 'vcores_millis_maps' in searches.
Unit
items per second
Parents
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_application_vcores_millis_reduces_rate

Description
Reduce CPU allocation. Called 'vcores_millis_reduces' in searches.
Unit
items per second
Parents
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_reports_containers_allocated_memory

Description
Memory allocated to YARN containers
Unit
MB seconds
Parents
cluster
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_reports_containers_allocated_vcores

Description
VCores allocated to YARN containers
Unit
VCore seconds
Parents
cluster
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_reports_containers_used_cpu

Description
CPU used by YARN containers
Unit
Percent seconds
Parents
cluster
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_reports_containers_used_memory

Description
Memory used by YARN containers
Unit
MB seconds
Parents
cluster
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_reports_containers_used_vcores

Description
VCores used by YARN containers
Unit
VCore seconds
Parents
cluster
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_reports_usage_aggregation_duration

Description
The duration for generating YARN usage reporting metrics by aggregating YARN container usage metrics
Unit
ms
Parents
cluster
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_reports_usage_apps_with_metadata

Description
YARN applications for which container usage was computed and also had metadata
Unit
Applications
Parents
cluster
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)

yarn_reports_usage_apps_without_metadata

Description
YARN applications for which container usage was computed but did not have metadata
Unit
Applications
Parents
cluster
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0)