TaskTracker Metrics

Reference information for TaskTracker Metrics

In addition to these base metrics, many aggregate metrics are available. If an entity type has parents defined, you can formulate all possible aggregate metrics using the formula base_metric_across_parents.

In addition, metrics for aggregate totals can be formed by adding the prefix total_ to the front of the metric name.

Use the type-ahead feature in the Cloudera Manager chart browser to find the exact aggregate metric name, in case the plural form does not end in "s".

For example, the following metric names may be valid for TaskTracker:

  • alerts_rate_across_clusters
  • total_alerts_rate_across_clusters

Some metrics, such as alerts_rate, apply to nearly every metric context. Others only apply to a certain service or role.

alerts_rate

Description
The number of alerts.
Unit
events per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

cgroup_cpu_system_rate

Description
CPU usage of the role's cgroup
Unit
seconds per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

cgroup_cpu_user_rate

Description
User Space CPU usage of the role's cgroup
Unit
seconds per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

cgroup_mem_page_cache

Description
Page cache usage of the role's cgroup
Unit
bytes
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

cgroup_mem_rss

Description
Resident memory of the role's cgroup
Unit
bytes
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

cgroup_mem_swap

Description
Swap usage of the role's cgroup
Unit
bytes
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

cgroup_read_bytes_rate

Description
Bytes read from all disks by the role's cgroup
Unit
bytes per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

cgroup_read_ios_rate

Description
Number of read I/O operations from all disks by the role's cgroup
Unit
ios per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

cgroup_write_bytes_rate

Description
Bytes written to all disks by the role's cgroup
Unit
bytes per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

cgroup_write_ios_rate

Description
Number of write I/O operations to all disks by the role's cgroup
Unit
ios per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

cpu_system_rate

Description
Total System CPU
Unit
seconds per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

cpu_system_with_descendants_rate

Description
The total system CPU time for this process and all its descendant processes
Unit
seconds per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

cpu_user_rate

Description
Total CPU user time
Unit
seconds per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

cpu_user_with_descendants_rate

Description
The total user CPU time for this process and all its descendant processes
Unit
seconds per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

events_critical_rate

Description
The number of critical events.
Unit
events per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

events_important_rate

Description
The number of important events.
Unit
events per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

events_informational_rate

Description
The number of informational events.
Unit
events per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

failed_dirs

Description
Failed Directories
Unit
directories
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

fd_max

Description
Maximum number of file descriptors
Unit
file descriptors
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

fd_open

Description
Open file descriptors.
Unit
file descriptors
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

health_bad_rate

Description
Percentage of Time with Bad Health
Unit
seconds per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

health_concerning_rate

Description
Percentage of Time with Concerning Health
Unit
seconds per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

health_disabled_rate

Description
Percentage of Time with Disabled Health
Unit
seconds per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

health_good_rate

Description
Percentage of Time with Good Health
Unit
seconds per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

health_unknown_rate

Description
Percentage of Time with Unknown Health
Unit
seconds per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

jvm_blocked_threads

Description
Blocked threads
Unit
threads
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

jvm_gc_rate

Description
Number of garbage collections
Unit
garbage collections per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

jvm_gc_time_ms_rate

Description
Total time spent garbage collecting.
Unit
ms per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

jvm_heap_committed_mb

Description
Total amount of committed heap memory.
Unit
MB
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

jvm_heap_used_mb

Description
Total amount of used heap memory.
Unit
MB
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

jvm_max_memory_mb

Description
Maximum allowed memory.
Unit
MB
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

jvm_new_threads

Description
New threads
Unit
threads
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

jvm_non_heap_committed_mb

Description
Total amount of committed non-heap memory.
Unit
MB
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

jvm_non_heap_used_mb

Description
Total amount of used non-heap memory.
Unit
MB
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

jvm_runnable_threads

Description
Runnable threads
Unit
threads
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

jvm_terminated_threads

Description
Terminated threads
Unit
threads
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

jvm_timed_waiting_threads

Description
Timed waiting threads
Unit
threads
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

jvm_total_threads

Description
Total threads
Unit
threads
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

jvm_waiting_threads

Description
Waiting threads
Unit
threads
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

log_error_rate

Description
Logged Errors
Unit
messages per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

log_fatal_rate

Description
Logged Fatals
Unit
messages per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

log_info_rate

Description
Logged Infos
Unit
messages per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

log_warn_rate

Description
Logged Warnings
Unit
messages per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

map_task_slots

Description
Map Task Slots
Unit
slots
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

maps_running

Description
Maps Running
Unit
tasks
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

mem_rss

Description
Resident memory used
Unit
bytes
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

mem_rss_with_descendants

Description
The total resident memory for this process and all its descendant processes
Unit
bytes
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

mem_swap

Description
Amount of swap memory used by this role's process.
Unit
bytes
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

mem_virtual

Description
Virtual memory used
Unit
bytes
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

mem_virtual_with_descendants

Description
The total virtual memory for this process and all its descendant processes
Unit
bytes
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

oom_exits_rate

Description
The number of times the role's backing process was killed due to an OutOfMemory error. This counter is only incremented if the Cloudera Manager "Kill When Out of Memory" option is enabled.
Unit
exits per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

read_bytes_rate

Description
The number of bytes read from the device
Unit
bytes per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

reduce_task_slots

Description
Reduce Task Slots
Unit
slots
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

reduces_running

Description
Reduces Running
Unit
tasks
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

shuffle_exceptions_caught_rate

Description
Shuffle Handler Exceptions Caught
Unit
exceptions per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

shuffle_failed_outputs_rate

Description
Shuffle Handler Failed Requests
Unit
requests per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

shuffle_handler_busy_percent

Description
Shuffle Handler Busy Percentage
Unit
percent
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

shuffle_output_bytes_rate

Description
Shuffle Output
Unit
bytes per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

shuffle_success_outputs_rate

Description
Shuffle Handler Successful Requests
Unit
requests per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

tasks_completed_rate

Description
Tasks Completed
Unit
tasks per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

tasks_failed_ping_rate

Description
Tasks Failed: Ping
Unit
tasks per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

tasks_failed_timeout_rate

Description
Tasks Failed: Timeout
Unit
tasks per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

unexpected_exits_rate

Description
The number of times the role's backing process exited unexpectedly.
Unit
exits per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

uptime

Description
For a host, the amount of time since the host was booted. For a role, the uptime of the backing process.
Unit
seconds
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]

web_metrics_collection_duration

Description
Web Server Responsiveness
Unit
ms
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0)

write_bytes_rate

Description
The number of bytes written to the device
Unit
bytes per second
Parents
cluster, mapreduce, rack
CDH Version
[CDH 5.0.0..CDH 6.0.0), [CDH 6.0.0..CDH 7.0.0), [CDH 7.0.0..CDH 8.0.0), [CM -1.0.0..CM -1.0.0]