HBase Properties in CDH 4.5.0
gatewaydefaultgroup
Advanced
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Deploy Directory | The directory where the client configs will be deployed | /etc/hbase | client_config_root_dir | true | |
| HBase Client Advanced Configuration Snippet (Safety Valve) for hbase-site.xml | For advanced use only, a string to be inserted into the client configuration for hbase-site.xml. | hbase_client_config_safety_valve | false | ||
| HBase Client Environment Advanced Configuration Snippet (Safety Valve) for hbase-env.sh | For advanced use only, key-value pairs (one on each line) to be inserted into the client configuration for hbase-env.sh | hbase_client_env_safety_valve | false | ||
| Client Java Configuration Options | These are Java command line arguments. Commonly, garbage collection flags or extra debugging flags would be passed here. | -XX:+HeapDumpOnOutOfMemoryError -XX:+UseConcMarkSweepGC -XX:-CMSConcurrentMTEnabled -XX:+CMSIncrementalMode -Djava.net.preferIPv4Stack=true | hbase_client_java_opts | false | |
| Gateway Logging Advanced Configuration Snippet (Safety Valve) | For advanced use only, a string to be inserted into log4j.properties for this role only. | log4j_safety_valve | false |
Logs
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Gateway Logging Threshold | The minimum log level for Gateway logs | INFO | log_threshold | false |
Monitoring
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Enable Configuration Change Alerts | When set, Cloudera Manager will send alerts when this entity's configuration changes. | false | enable_config_alerts | false |
Other
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Alternatives Priority | The priority level that the client configuration will have in the Alternatives system on the hosts. Higher priority levels will cause Alternatives to prefer this configuration over any others. | 90 | client_config_priority | true |
hbaserestserverdefaultgroup
Advanced
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| HBase REST Server Environment Advanced Configuration Snippet (Safety Valve) | For advanced use only, key-value pairs (one on each line) to be inserted into a role's environment. Applies to configurations of this role except client configuration. | HBASERESTSERVER_role_env_safety_valve | false | ||
| HBase REST Server Advanced Configuration Snippet (Safety Valve) for hbase-site.xml | For advanced use only, a string to be inserted into hbase-site.xml for this role only. | hbase_restserver_config_safety_valve | false | ||
| Java Configuration Options for HBase REST Server | These arguments will be passed as part of the Java command line. Commonly, garbage collection flags or extra debugging flags would be passed here. | -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:-CMSConcurrentMTEnabled -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled | hbase_restserver_java_opts | false | |
| HBase REST Server Logging Advanced Configuration Snippet (Safety Valve) | For advanced use only, a string to be inserted into log4j.properties for this role only. | log4j_safety_valve | false | ||
| Heap Dump Directory | Path to directory where heap dumps are generated when java.lang.OutOfMemoryError error is thrown. This directory is automatically created if it does not exist. If this directory already exists, role user must have write access to this directory. If this directory is shared among multiple roles, it should have 1777 permissions. The heap dump files are created with 600 permissions and are owned by the role user. The amount of free space in this directory should be greater than the maximum Java Process heap size configured for this role. | oom_heap_dump_dir | /tmp | oom_heap_dump_dir | false |
| Dump Heap When Out of Memory | When set, generates heap dump file when java.lang.OutOfMemoryError is thrown. | false | oom_heap_dump_enabled | true | |
| Kill When Out of Memory | When set, a SIGKILL signal is sent to the role process when java.lang.OutOfMemoryError is thrown. | true | oom_sigkill_enabled | true | |
| Automatically Restart Process | When set, this role's process is automatically (and transparently) restarted in the event of an unexpected failure. | false | process_auto_restart | true |
Logs
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| HBase REST Server Log Directory | Directory where HBase REST Server will place its log files. | hadoop.log.dir | /var/log/hbase | hbase_restserver_log_dir | false |
| HBase REST Server Logging Threshold | The minimum log level for HBase REST Server logs | INFO | log_threshold | false | |
| HBase REST Server Maximum Log File Backups | The maximum number of rolled log files to keep for HBase REST Server logs. Typically used by log4j or logback. | 10 | max_log_backup_index | false | |
| HBase REST Server Max Log Size | The maximum size, in megabytes, per log file for HBase REST Server logs. Typically used by log4j or logback. | 200 MiB | max_log_size | false |
Monitoring
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Enable Health Alerts for this Role | When set, Cloudera Manager will send alerts when the health of this role reaches the threshold specified by the EventServer setting eventserver_health_events_alert_threshold | true | enable_alerts | false | |
| Enable Configuration Change Alerts | When set, Cloudera Manager will send alerts when this entity's configuration changes. | false | enable_config_alerts | false | |
| File Descriptor Monitoring Thresholds | The health test thresholds of the number of file descriptors used. Specified as a percentage of file descriptor limit. | Warning: 50.0 %, Critical: 70.0 % | hbaserestserver_fd_thresholds | false | |
| HBase REST Server Host Health Test | When computing the overall HBase REST Server health, consider the host's health. | true | hbaserestserver_host_health_enabled | false | |
| HBase REST Server Process Health Test | Enables the health test that the HBase REST Server's process state is consistent with the role configuration | true | hbaserestserver_scm_health_enabled | false | |
| Heap Dump Directory Free Space Monitoring Absolute Thresholds | The health test thresholds for monitoring of free space on the filesystem that contains this role's heap dump directory. | Warning: 10 GiB, Critical: 5 GiB | heap_dump_directory_free_space_absolute_thresholds | false | |
| Heap Dump Directory Free Space Monitoring Percentage Thresholds | The health test thresholds for monitoring of free space on the filesystem that contains this role's heap dump directory. Specified as a percentage of the capacity on that filesystem. This setting is not used if a Heap Dump Directory Free Space Monitoring Absolute Thresholds setting is configured. | Warning: Never, Critical: Never | heap_dump_directory_free_space_percentage_thresholds | false | |
| Log Directory Free Space Monitoring Absolute Thresholds | The health test thresholds for monitoring of free space on the filesystem that contains this role's log directory. | Warning: 10 GiB, Critical: 5 GiB | log_directory_free_space_absolute_thresholds | false | |
| Log Directory Free Space Monitoring Percentage Thresholds | The health test thresholds for monitoring of free space on the filesystem that contains this role's log directory. Specified as a percentage of the capacity on that filesystem. This setting is not used if a Log Directory Free Space Monitoring Absolute Thresholds setting is configured. | Warning: Never, Critical: Never | log_directory_free_space_percentage_thresholds | false | |
| Rules to Extract Events from Log Files | This file contains the rules which govern how log messages are turned into events by the custom log4j appender that this role loads.
It is in JSON format, and is composed of a list of rules. Every log message is evaluated against each of these rules in turn to decide whether or not to send an event for that message. Each rule has
some or all of the following fields:
|
version: 0, rules: [ alert: false, rate: 1, periodminutes: 1, threshold:FATAL, alert: false, rate: 1, periodminutes: 2, exceptiontype: .*, alert: false, rate: 1, periodminutes: 1, threshold:WARN ] | log_event_whitelist | false | |
| Process Swap Memory Thresholds | The health test thresholds on the swap memory usage of the process. | Warning: Any, Critical: Never | process_swap_memory_thresholds | false | |
| Role Triggers | The configured triggers for this role. This is a JSON formatted list of triggers. These triggers are evaluated as part as the health
system. Every trigger expression is parsed, and if the trigger condition is met, the list of actions provided in the trigger expression is executed. Each trigger has all of the following fields:
|
[] | role_triggers | true | |
| Unexpected Exits Thresholds | The health test thresholds for unexpected exits encountered within a recent period specified by the unexpected_exits_window configuration for the role. | Warning: Never, Critical: Any | unexpected_exits_thresholds | false | |
| Unexpected Exits Monitoring Period | The period to review when computing unexpected exits. | 5 minute(s) | unexpected_exits_window | false |
Other
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Enable HBase REST Server Read Only Mode | When false, all HTTP methods are permitted (GET/PUT/POST/DELETE). When true, only GET is permitted. | hbase.rest.readonly | false | hbase_restserver_readonly | false |
Performance
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| HBase REST Server Maximum Threads | Maximum size of the HBase REST Server thread pool. The server can process this number of concurrent requests. Setting this too high can lead to out of memory errors. | hbase.rest.threads.max | 100 | hbase_restserver_threads_max | true |
| HBase REST Server Minimum Threads | Minimum size of the HBase REST Server thread pool. The server will maintain at least this number of threads in the pool at all times. The thread pool can grow up to the maximum size set by hbase.rest.threads.max. | hbase.rest.threads.min | 2 | hbase_restserver_threads_min | true |
| Maximum Process File Descriptors | If configured, overrides the process soft and hard rlimits (also called ulimits) for file descriptors to the configured value. | rlimit_fds | false |
Ports and Addresses
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| HBase REST Server DNS Network Interface | The name of the DNS network interface from which an HBase REST Server should report its IP address. | hbase.rest.dns.interface | hbase_restserver_dns_interface | false | |
| HBase REST Server DNS Name Server | The host name or IP address of the DNS name server which an HBase REST Server should use to determine the host name used for communication and display purposes. | hbase.rest.dns.nameserver | hbase_restserver_dns_nameserver | false | |
| HBase REST Server Host Address | HBase REST Server will bind to this address. | hbase.rest.host | 0.0.0.0 | hbase_restserver_host | false |
| HBase REST Server Web UI Bind to Wildcard Address | If true, HBase REST Server Web UI will bind to a wildcard address (0.0.0.0). Otherwise it will bind to a host name. Only available in CDH 4.3 and later. | hbase.rest.info.bindAddress | true | hbase_restserver_info_bind_to_wildcard | false |
| HBase REST Server Web UI Port | The port that HBase REST Server Web UI binds to. | hbase.rest.info.port | 8085 | hbase_restserver_info_port | true |
| HBase REST Server Port | The port that HBase REST Server binds to. | hbase.rest.port | 20550 | hbase_restserver_port | true |
Resource Management
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Java Heap Size of HBase REST Server in Bytes | Maximum size in bytes for the Java Process heap memory. Passed to Java -Xmx. | 1 GiB | hbase_restserver_java_heapsize | false | |
| Cgroup CPU Shares | Number of CPU shares to assign to this role. The greater the number of shares, the larger the share of the host's CPUs that will be given to this role when the host experiences CPU contention. Must be between 2 and 262144. Defaults to 1024 for processes not managed by Cloudera Manager. | cpu.shares | 1024 | rm_cpu_shares | true |
| Cgroup I/O Weight | Weight for the read I/O requests issued by this role. The greater the weight, the higher the priority of the requests when the host experiences I/O contention. Must be between 100 and 1000. Defaults to 1000 for processes not managed by Cloudera Manager. | blkio.weight | 500 | rm_io_weight | true |
| Cgroup Memory Hard Limit | Hard memory limit to assign to this role, enforced by the Linux kernel. When the limit is reached, the kernel will reclaim pages charged to the process. If reclaiming fails, the kernel may kill the process. Both anonymous as well as page cache pages contribute to the limit. Use a value of -1 B to specify no limit. By default processes not managed by Cloudera Manager will have no limit. | memory.limit_in_bytes | -1 MiB | rm_memory_hard_limit | true |
| Cgroup Memory Soft Limit | Soft memory limit to assign to this role, enforced by the Linux kernel. When the limit is reached, the kernel will reclaim pages charged to the process if and only if the host is facing memory pressure. If reclaiming fails, the kernel may kill the process. Both anonymous as well as page cache pages contribute to the limit. Use a value of -1 B to specify no limit. By default processes not managed by Cloudera Manager will have no limit. | memory.soft_limit_in_bytes | -1 MiB | rm_memory_soft_limit | true |
Stacks Collection
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Stacks Collection Data Retention | The amount of stacks data that is retained. After the retention limit is reached, the oldest data is deleted. | stacks_collection_data_retention | 100 MiB | stacks_collection_data_retention | false |
| Stacks Collection Directory | The directory in which stacks logs are placed. If not set, stacks are logged into a stacks subdirectory of the role's log directory. | stacks_collection_directory | stacks_collection_directory | false | |
| Stacks Collection Enabled | Whether or not periodic stacks collection is enabled. | stacks_collection_enabled | false | stacks_collection_enabled | true |
| Stacks Collection Frequency | The frequency with which stacks are collected. | stacks_collection_frequency | 5.0 second(s) | stacks_collection_frequency | false |
| Stacks Collection Method | The method used to collect stacks. The jstack option involves periodically running the jstack command against the role's daemon process. The servlet method is available for those roles that have an HTTP server endpoint exposing the current stacks traces of all threads. When the servlet method is selected, that HTTP endpoint is periodically scraped. | stacks_collection_method | jstack | stacks_collection_method | false |
hbasethriftserverdefaultgroup
Advanced
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| HBase Thrift Server Environment Advanced Configuration Snippet (Safety Valve) | For advanced use only, key-value pairs (one on each line) to be inserted into a role's environment. Applies to configurations of this role except client configuration. | HBASETHRIFTSERVER_role_env_safety_valve | false | ||
| HBase Thrift Server Advanced Configuration Snippet (Safety Valve) for hbase-site.xml | For advanced use only, a string to be inserted into hbase-site.xml for this role only. | hbase_thriftserver_config_safety_valve | false | ||
| Java Configuration Options for HBase Thrift Server | These arguments will be passed as part of the Java command line. Commonly, garbage collection flags or extra debugging flags would be passed here. | -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:-CMSConcurrentMTEnabled -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled | hbase_thriftserver_java_opts | false | |
| HBase Thrift Server Logging Advanced Configuration Snippet (Safety Valve) | For advanced use only, a string to be inserted into log4j.properties for this role only. | log4j_safety_valve | false | ||
| Heap Dump Directory | Path to directory where heap dumps are generated when java.lang.OutOfMemoryError error is thrown. This directory is automatically created if it does not exist. If this directory already exists, role user must have write access to this directory. If this directory is shared among multiple roles, it should have 1777 permissions. The heap dump files are created with 600 permissions and are owned by the role user. The amount of free space in this directory should be greater than the maximum Java Process heap size configured for this role. | oom_heap_dump_dir | /tmp | oom_heap_dump_dir | false |
| Dump Heap When Out of Memory | When set, generates heap dump file when java.lang.OutOfMemoryError is thrown. | false | oom_heap_dump_enabled | true | |
| Kill When Out of Memory | When set, a SIGKILL signal is sent to the role process when java.lang.OutOfMemoryError is thrown. | true | oom_sigkill_enabled | true | |
| Automatically Restart Process | When set, this role's process is automatically (and transparently) restarted in the event of an unexpected failure. | false | process_auto_restart | true |
Logs
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| HBase Thrift Server Log Directory | Directory where HBase Thrift Server will place its log files. | hadoop.log.dir | /var/log/hbase | hbase_thriftserver_log_dir | false |
| HBase Thrift Server Logging Threshold | The minimum log level for HBase Thrift Server logs | INFO | log_threshold | false | |
| HBase Thrift Server Maximum Log File Backups | The maximum number of rolled log files to keep for HBase Thrift Server logs. Typically used by log4j or logback. | 10 | max_log_backup_index | false | |
| HBase Thrift Server Max Log Size | The maximum size, in megabytes, per log file for HBase Thrift Server logs. Typically used by log4j or logback. | 200 MiB | max_log_size | false |
Monitoring
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Enable Health Alerts for this Role | When set, Cloudera Manager will send alerts when the health of this role reaches the threshold specified by the EventServer setting eventserver_health_events_alert_threshold | true | enable_alerts | false | |
| Enable Configuration Change Alerts | When set, Cloudera Manager will send alerts when this entity's configuration changes. | false | enable_config_alerts | false | |
| File Descriptor Monitoring Thresholds | The health test thresholds of the number of file descriptors used. Specified as a percentage of file descriptor limit. | Warning: 50.0 %, Critical: 70.0 % | hbasethriftserver_fd_thresholds | false | |
| HBase Thrift Server Host Health Test | When computing the overall HBase Thrift Server health, consider the host's health. | true | hbasethriftserver_host_health_enabled | false | |
| HBase Thrift Server Process Health Test | Enables the health test that the HBase Thrift Server's process state is consistent with the role configuration | true | hbasethriftserver_scm_health_enabled | false | |
| Heap Dump Directory Free Space Monitoring Absolute Thresholds | The health test thresholds for monitoring of free space on the filesystem that contains this role's heap dump directory. | Warning: 10 GiB, Critical: 5 GiB | heap_dump_directory_free_space_absolute_thresholds | false | |
| Heap Dump Directory Free Space Monitoring Percentage Thresholds | The health test thresholds for monitoring of free space on the filesystem that contains this role's heap dump directory. Specified as a percentage of the capacity on that filesystem. This setting is not used if a Heap Dump Directory Free Space Monitoring Absolute Thresholds setting is configured. | Warning: Never, Critical: Never | heap_dump_directory_free_space_percentage_thresholds | false | |
| Log Directory Free Space Monitoring Absolute Thresholds | The health test thresholds for monitoring of free space on the filesystem that contains this role's log directory. | Warning: 10 GiB, Critical: 5 GiB | log_directory_free_space_absolute_thresholds | false | |
| Log Directory Free Space Monitoring Percentage Thresholds | The health test thresholds for monitoring of free space on the filesystem that contains this role's log directory. Specified as a percentage of the capacity on that filesystem. This setting is not used if a Log Directory Free Space Monitoring Absolute Thresholds setting is configured. | Warning: Never, Critical: Never | log_directory_free_space_percentage_thresholds | false | |
| Rules to Extract Events from Log Files | This file contains the rules which govern how log messages are turned into events by the custom log4j appender that this role loads.
It is in JSON format, and is composed of a list of rules. Every log message is evaluated against each of these rules in turn to decide whether or not to send an event for that message. Each rule has
some or all of the following fields:
|
version: 0, rules: [ alert: false, rate: 1, periodminutes: 1, threshold:FATAL, alert: false, rate: 1, periodminutes: 2, exceptiontype: .*, alert: false, rate: 1, periodminutes: 1, threshold:WARN ] | log_event_whitelist | false | |
| Process Swap Memory Thresholds | The health test thresholds on the swap memory usage of the process. | Warning: Any, Critical: Never | process_swap_memory_thresholds | false | |
| Role Triggers | The configured triggers for this role. This is a JSON formatted list of triggers. These triggers are evaluated as part as the health
system. Every trigger expression is parsed, and if the trigger condition is met, the list of actions provided in the trigger expression is executed. Each trigger has all of the following fields:
|
[] | role_triggers | true | |
| Unexpected Exits Thresholds | The health test thresholds for unexpected exits encountered within a recent period specified by the unexpected_exits_window configuration for the role. | Warning: Never, Critical: Any | unexpected_exits_thresholds | false | |
| Unexpected Exits Monitoring Period | The period to review when computing unexpected exits. | 5 minute(s) | unexpected_exits_window | false |
Other
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Enable HBase Thrift Server Compact Protocol | Use the TCompactProtocol instead of the default TBinaryProtocol. TCompactProtocol is a binary protocol that is more compact than the default and typically more efficient. | hbase.regionserver.thrift.compact | false | hbase_thriftserver_compact | false |
| Enable HBase Thrift Server Framed Transport | Use framed transport. When using the THsHaServer or TNonblockingServer, framed transport is always used irrespective of this configuration value. | hbase.regionserver.thrift.framed | false | hbase_thriftserver_framed | false |
| HBase Thrift Server Type | Type of HBase Thrift Server. | hbase.regionserver.thrift.server.type | threadpool | hbase_thriftserver_type | false |
Performance
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| HBase Thrift Server Min Worker Threads | The "core size" of the thread pool. New threads are created on every connection until this many threads are created. | hbase.thrift.minWorkerThreads | 200 | hbase_thriftserver_min_worker_threads | false |
| Maximum Process File Descriptors | If configured, overrides the process soft and hard rlimits (also called ulimits) for file descriptors to the configured value. | rlimit_fds | false |
Ports and Addresses
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| HBase Thrift Server Bind Address | Address to bind the HBase Thrift Server to. When using the THsHaServer or the TNonblockingServer, always binds to 0.0.0.0 irrespective of this configuration value. | hbase.regionserver.thrift.ipaddress | 0.0.0.0 | hbase_thriftserver_bindaddress | false |
| HBase Thrift Server DNS Network Interface | The name of the DNS network interface from which an HBase Thrift Server should report its IP address. | hbase.thrift.dns.interface | hbase_thriftserver_dns_interface | false | |
| HBase Thrift Server DNS Name Server | The host name or IP address of the DNS name server which an HBase Thrift Server should use to determine the host name used for communication and display purposes. | hbase.thrift.dns.nameserver | hbase_thriftserver_dns_nameserver | false | |
| HBase Thrift Server Web UI Bind to Wildcard Address | If true, HBase Thrift Server Web UI will bind to a wildcard address (0.0.0.0). Otherwise it will bind to a host name. Only available in CDH 4.3 and later. | hbase.thrift.info.bindAddress | true | hbase_thriftserver_info_bind_to_wildcard | false |
| HBase Thrift Server Web UI Port | The port that HBase Thrift Server Web UI binds to. | hbase.thrift.info.port | 9095 | hbase_thriftserver_info_port | true |
| HBase Thrift Server Port | The port that HBase Thrift Server binds to. | hbase.regionserver.thrift.port | 9090 | hbase_thriftserver_port | true |
Resource Management
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Java Heap Size of HBase Thrift Server in Bytes | Maximum size in bytes for the Java Process heap memory. Passed to Java -Xmx. | 1 GiB | hbase_thriftserver_java_heapsize | false | |
| Cgroup CPU Shares | Number of CPU shares to assign to this role. The greater the number of shares, the larger the share of the host's CPUs that will be given to this role when the host experiences CPU contention. Must be between 2 and 262144. Defaults to 1024 for processes not managed by Cloudera Manager. | cpu.shares | 1024 | rm_cpu_shares | true |
| Cgroup I/O Weight | Weight for the read I/O requests issued by this role. The greater the weight, the higher the priority of the requests when the host experiences I/O contention. Must be between 100 and 1000. Defaults to 1000 for processes not managed by Cloudera Manager. | blkio.weight | 500 | rm_io_weight | true |
| Cgroup Memory Hard Limit | Hard memory limit to assign to this role, enforced by the Linux kernel. When the limit is reached, the kernel will reclaim pages charged to the process. If reclaiming fails, the kernel may kill the process. Both anonymous as well as page cache pages contribute to the limit. Use a value of -1 B to specify no limit. By default processes not managed by Cloudera Manager will have no limit. | memory.limit_in_bytes | -1 MiB | rm_memory_hard_limit | true |
| Cgroup Memory Soft Limit | Soft memory limit to assign to this role, enforced by the Linux kernel. When the limit is reached, the kernel will reclaim pages charged to the process if and only if the host is facing memory pressure. If reclaiming fails, the kernel may kill the process. Both anonymous as well as page cache pages contribute to the limit. Use a value of -1 B to specify no limit. By default processes not managed by Cloudera Manager will have no limit. | memory.soft_limit_in_bytes | -1 MiB | rm_memory_soft_limit | true |
Stacks Collection
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Stacks Collection Data Retention | The amount of stacks data that is retained. After the retention limit is reached, the oldest data is deleted. | stacks_collection_data_retention | 100 MiB | stacks_collection_data_retention | false |
| Stacks Collection Directory | The directory in which stacks logs are placed. If not set, stacks are logged into a stacks subdirectory of the role's log directory. | stacks_collection_directory | stacks_collection_directory | false | |
| Stacks Collection Enabled | Whether or not periodic stacks collection is enabled. | stacks_collection_enabled | false | stacks_collection_enabled | true |
| Stacks Collection Frequency | The frequency with which stacks are collected. | stacks_collection_frequency | 5.0 second(s) | stacks_collection_frequency | false |
| Stacks Collection Method | The method used to collect stacks. The jstack option involves periodically running the jstack command against the role's daemon process. The servlet method is available for those roles that have an HTTP server endpoint exposing the current stacks traces of all threads. When the servlet method is selected, that HTTP endpoint is periodically scraped. | stacks_collection_method | jstack | stacks_collection_method | false |
masterdefaultgroup
Advanced
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Master Environment Advanced Configuration Snippet (Safety Valve) | For advanced use only, key-value pairs (one on each line) to be inserted into a role's environment. Applies to configurations of this role except client configuration. | MASTER_role_env_safety_valve | false | ||
| Hadoop Metrics Advanced Configuration Snippet (Safety Valve) | Advanced Configuration Snippet (Safety Valve) for Hadoop Metrics. Properties will be inserted into hadoop-metrics.properties for this role only. Note that Cloudera Manager tunes hadoop-metrics.properties to work optimally with its Service Monitoring features. By overriding the default, Cloudera Manager might not be able to provide accurate monitoring information, health tests or alerts. | hadoop_metrics_safety_valve | false | ||
| HBase Coprocessor Master Classes | List of org.apache.hadoop.hbase.coprocessor.MasterObserver coprocessors that are loaded by default on the active HMaster process. For any implemented coprocessor methods, the listed classes will be called in order. After implementing your own MasterObserver, just put it in HBase's classpath and add the fully qualified class name here. | hbase.coprocessor.master.classes | hbase_coprocessor_master_classes | false | |
| Master Advanced Configuration Snippet (Safety Valve) for hbase-site.xml | For advanced use only, a string to be inserted into hbase-site.xml for this role only. | hbase_master_config_safety_valve | false | ||
| Java Configuration Options for HBase Master | These arguments will be passed as part of the Java command line. Commonly, garbage collection flags or extra debugging flags would be passed here. | -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:-CMSConcurrentMTEnabled -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled | hbase_master_java_opts | false | |
| Master Logging Advanced Configuration Snippet (Safety Valve) | For advanced use only, a string to be inserted into log4j.properties for this role only. | log4j_safety_valve | false | ||
| Heap Dump Directory | Path to directory where heap dumps are generated when java.lang.OutOfMemoryError error is thrown. This directory is automatically created if it does not exist. If this directory already exists, role user must have write access to this directory. If this directory is shared among multiple roles, it should have 1777 permissions. The heap dump files are created with 600 permissions and are owned by the role user. The amount of free space in this directory should be greater than the maximum Java Process heap size configured for this role. | oom_heap_dump_dir | /tmp | oom_heap_dump_dir | false |
| Dump Heap When Out of Memory | When set, generates heap dump file when java.lang.OutOfMemoryError is thrown. | false | oom_heap_dump_enabled | true | |
| Kill When Out of Memory | When set, a SIGKILL signal is sent to the role process when java.lang.OutOfMemoryError is thrown. | true | oom_sigkill_enabled | true | |
| Automatically Restart Process | When set, this role's process is automatically (and transparently) restarted in the event of an unexpected failure. | false | process_auto_restart | true |
Logs
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Master Log Directory | Directory where Master will place its log files. | hadoop.log.dir | /var/log/hbase | hbase_master_log_dir | false |
| Master Logging Threshold | The minimum log level for Master logs | INFO | log_threshold | false | |
| Master Maximum Log File Backups | The maximum number of rolled log files to keep for Master logs. Typically used by log4j or logback. | 10 | max_log_backup_index | false | |
| Master Max Log Size | The maximum size, in megabytes, per log file for Master logs. Typically used by log4j or logback. | 200 MiB | max_log_size | false |
Metrics
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Hadoop Metrics Class | Implementation daemons will use to report some internal statistics. The default (NoEmitMetricsContext) will display metrics on /metrics on the status port. The GangliaContext and GangliaContext31 classes will report metrics to your specified Ganglia Monitoring Daemons (gmond). The ganglia wire format changed incompatibly at version 3.1.0. If you are running any version of ganglia 3.1.0 or newer, use the GangliaContext31 metric class; otherwise, use the GangliaContext metric class. | org.apache.hadoop.metrics.spi.NoEmitMetricsContext | hadoop_metrics_class | false | |
| Hadoop Metrics Output Directory | If using FileContext, directory to write metrics to. | /tmp/metrics | hadoop_metrics_dir | false | |
| Hadoop Metrics Ganglia Servers | If using GangliaContext, a comma-delimited list of host:port pairs pointing to 'gmond' servers you would like to publish metrics to. In practice, this set of 'gmond' should match the set of 'gmond' in your 'gmetad' datasource list for the cluster. | hadoop_metrics_ganglia_servers | false | ||
| Extended Period | Time period in seconds to reset long-running metrics (e.g. compactions). This is an HBase specific configuration. | hbase.extendedperiod | 1 hour(s) | hbase_metrics_extended_period | false |
Monitoring
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Enable Health Alerts for this Role | When set, Cloudera Manager will send alerts when the health of this role reaches the threshold specified by the EventServer setting eventserver_health_events_alert_threshold | true | enable_alerts | false | |
| Enable Configuration Change Alerts | When set, Cloudera Manager will send alerts when this entity's configuration changes. | false | enable_config_alerts | false | |
| Heap Dump Directory Free Space Monitoring Absolute Thresholds | The health test thresholds for monitoring of free space on the filesystem that contains this role's heap dump directory. | Warning: 10 GiB, Critical: 5 GiB | heap_dump_directory_free_space_absolute_thresholds | false | |
| Heap Dump Directory Free Space Monitoring Percentage Thresholds | The health test thresholds for monitoring of free space on the filesystem that contains this role's heap dump directory. Specified as a percentage of the capacity on that filesystem. This setting is not used if a Heap Dump Directory Free Space Monitoring Absolute Thresholds setting is configured. | Warning: Never, Critical: Never | heap_dump_directory_free_space_percentage_thresholds | false | |
| Log Directory Free Space Monitoring Absolute Thresholds | The health test thresholds for monitoring of free space on the filesystem that contains this role's log directory. | Warning: 10 GiB, Critical: 5 GiB | log_directory_free_space_absolute_thresholds | false | |
| Log Directory Free Space Monitoring Percentage Thresholds | The health test thresholds for monitoring of free space on the filesystem that contains this role's log directory. Specified as a percentage of the capacity on that filesystem. This setting is not used if a Log Directory Free Space Monitoring Absolute Thresholds setting is configured. | Warning: Never, Critical: Never | log_directory_free_space_percentage_thresholds | false | |
| Rules to Extract Events from Log Files | This file contains the rules which govern how log messages are turned into events by the custom log4j appender that this role loads.
It is in JSON format, and is composed of a list of rules. Every log message is evaluated against each of these rules in turn to decide whether or not to send an event for that message. Each rule has
some or all of the following fields:
|
version: 0, rules: [ alert: false, rate: 1, periodminutes: 1, threshold:FATAL, alert: false, rate: 0, exceptiontype: java.io.IOException, alert: false, rate: 0, exceptiontype: java.net.SocketException, alert: false, rate: 0, exceptiontype: java.net.SocketClosedException, alert: false, rate: 0, exceptiontype: java.io.EOFException, alert: false, rate: 0, exceptiontype: java.nio.channels.CancelledKeyException, alert: false, rate: 0, threshold:WARN, content: .* is deprecated. Instead, use .*, alert: false, rate: 0, threshold:WARN, content: .* is deprecated. Use .* instead, alert: false, rate: 1, periodminutes: 1, threshold:WARN, content: IPC Server handler.*ClosedChannelException, alert: false, rate: 1, periodminutes: 1, threshold:WARN, content: IPC Server Responder, call.*output error, alert: false, rate: 1, periodminutes: 1, threshold:WARN, content: Daughter regiondir does not exist: .*, alert: false, rate: 1, periodminutes: 1, threshold:WARN, content: File.*might still be open.*, alert: false, rate: 1, periodminutes: 1, threshold:WARN, content: File.*might still be open.*, alert: false, rate: 1, periodminutes: 1, threshold:WARN, content: Moving table .+ state to enabled but was already enabled, alert: false, rate: 1, periodminutes: 1, threshold:WARN, content: Received OPENED for region.*but region was in the state.*, alert: false, rate: 1, periodminutes: 2, exceptiontype: .*, alert: false, rate: 0, threshold:WARN, content:Unknown job [^ ]+ being deleted.*, alert: false, rate: 0, threshold:WARN, content:Error executing shell command .+ No such process.+, alert: false, rate: 0, threshold:WARN, content:.*attempt to override final parameter.+, alert: false, rate: 0, threshold:WARN, content:[^ ]+ is a deprecated filesystem name. Use.*, alert: false, rate: 1, periodminutes: 1, threshold:WARN ] | log_event_whitelist | false | |
| HBase Master Canary Health Test | Enables the health test that a client can connect to the HBase Master | true | master_canary_health_enabled | false | |
| File Descriptor Monitoring Thresholds | The health test thresholds of the number of file descriptors used. Specified as a percentage of file descriptor limit. | Warning: 50.0 %, Critical: 70.0 % | master_fd_thresholds | false | |
| Garbage Collection Duration Thresholds | The health test thresholds for the weighted average time spent in Java garbage collection. Specified as a percentage of elapsed wall clock time. | Warning: 30.0, Critical: 60.0 | master_gc_duration_thresholds | false | |
| Garbage Collection Duration Monitoring Period | The period to review when computing the moving average of garbage collection time. | 5 minute(s) | master_gc_duration_window | false | |
| Master Host Health Test | When computing the overall Master health, consider the host's health. | true | master_host_health_enabled | false | |
| Master Process Health Test | Enables the health test that the Master's process state is consistent with the role configuration | true | master_scm_health_enabled | false | |
| Health Check Startup Tolerance | The amount of time allowed after this role is started that failures of health checks that rely on communication with this role will be tolerated. | 5 minute(s) | master_startup_tolerance | false | |
| Web Metric Collection | Enables the health test that the Cloudera Manager Agent can successfully contact and gather metrics from the web server. | true | master_web_metric_collection_enabled | false | |
| Web Metric Collection Duration | The health test thresholds on the duration of the metrics request to the web server. | Warning: 10 second(s), Critical: Never | master_web_metric_collection_thresholds | false | |
| Process Swap Memory Thresholds | The health test thresholds on the swap memory usage of the process. | Warning: Any, Critical: Never | process_swap_memory_thresholds | false | |
| Role Triggers | The configured triggers for this role. This is a JSON formatted list of triggers. These triggers are evaluated as part as the health
system. Every trigger expression is parsed, and if the trigger condition is met, the list of actions provided in the trigger expression is executed. Each trigger has all of the following fields:
|
[] | role_triggers | true | |
| Unexpected Exits Thresholds | The health test thresholds for unexpected exits encountered within a recent period specified by the unexpected_exits_window configuration for the role. | Warning: Never, Critical: Any | unexpected_exits_thresholds | false | |
| Unexpected Exits Monitoring Period | The period to review when computing unexpected exits. | 5 minute(s) | unexpected_exits_window | false |
Other
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| HBase Master Handler Count | Number of RPC Server instances spun up on HBase Master. | hbase.master.handler.count | 25 | hbase_master_handler_count | false |
| HBase Master Log Cleaner Plugins | A comma-separated list of LogCleanerDelegate(s) that are used in LogsCleaner. WAL/HLog cleaner(s) are called in order, so put the log cleaner that prunes the most log files in the front. To implement your own LogCleanerDelegate, add it to HBase's classpath and add the fully-qualified class name here. You should always add the above default log cleaners in the list, unless you have a special reason not to. | hbase.master.logcleaner.plugins | hbase_master_logcleaner_plugins | false | |
| Maximum Time to Keep HLogs | Maximum time an HLog remains in the .oldlogdir directory until an HBase Master thread deletes it. | hbase.master.logcleaner.ttl | 1 minute(s) | hbase_master_logcleaner_ttl | false |
Performance
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Region Closing Threads | Number of pooled threads to handle region closing in the master. | hbase.master.executor.closeregion.threads | 5 | hbase_master_executor_closeregion_threads | false |
| Region Opening Threads | Number of pooled threads to handle region opening in the master. | hbase.master.executor.openregion.threads | 5 | hbase_master_executor_openregion_threads | false |
| RegionServer Recovery Threads | Number of pooled threads to handle the recovery of the region servers in the master. | hbase.master.executor.serverops.threads | 5 | hbase_master_executor_serverops_threads | false |
| Maximum Process File Descriptors | If configured, overrides the process soft and hard rlimits (also called ulimits) for file descriptors to the configured value. | rlimit_fds | false |
Ports and Addresses
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| HBase Master Bind to Wildcard Address | When true, HBase Master will bind to 0.0.0.0. Only available with CDH 4.3 and later. | hbase.master.ipc.address | true | hbase_master_bind_to_wildcard_address | true |
| HBase Master DNS Network Interface | The name of the DNS network interface from which an HBase Master should report its IP address. | hbase.master.dns.interface | hbase_master_dns_interface | false | |
| HBase Master DNS Name Server | The host name or IP address of the DNS name server which an HBase Master should use to determine the host name used for communication and display purposes. | hbase.master.dns.nameserver | hbase_master_dns_nameserver | false | |
| HBase Master Web UI Address | The address for the HBase Master web UI | hbase.master.info.bindAddress | hbase_master_info_bindAddress | false | |
| HBase Master Web UI Port | The port for the HBase Master web UI. Set to -1 to disable the HBase Master web UI. | hbase.master.info.port | 60010 | hbase_master_info_port | false |
| HBase Master Port | The port that the HBase Master binds to. | hbase.master.port | 60000 | hbase_master_port | false |
Resource Management
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Java Heap Size of HBase Master in Bytes | Maximum size in bytes for the Java Process heap memory. Passed to Java -Xmx. | 1 GiB | hbase_master_java_heapsize | false | |
| Cgroup CPU Shares | Number of CPU shares to assign to this role. The greater the number of shares, the larger the share of the host's CPUs that will be given to this role when the host experiences CPU contention. Must be between 2 and 262144. Defaults to 1024 for processes not managed by Cloudera Manager. | cpu.shares | 1024 | rm_cpu_shares | true |
| Cgroup I/O Weight | Weight for the read I/O requests issued by this role. The greater the weight, the higher the priority of the requests when the host experiences I/O contention. Must be between 100 and 1000. Defaults to 1000 for processes not managed by Cloudera Manager. | blkio.weight | 500 | rm_io_weight | true |
| Cgroup Memory Hard Limit | Hard memory limit to assign to this role, enforced by the Linux kernel. When the limit is reached, the kernel will reclaim pages charged to the process. If reclaiming fails, the kernel may kill the process. Both anonymous as well as page cache pages contribute to the limit. Use a value of -1 B to specify no limit. By default processes not managed by Cloudera Manager will have no limit. | memory.limit_in_bytes | -1 MiB | rm_memory_hard_limit | true |
| Cgroup Memory Soft Limit | Soft memory limit to assign to this role, enforced by the Linux kernel. When the limit is reached, the kernel will reclaim pages charged to the process if and only if the host is facing memory pressure. If reclaiming fails, the kernel may kill the process. Both anonymous as well as page cache pages contribute to the limit. Use a value of -1 B to specify no limit. By default processes not managed by Cloudera Manager will have no limit. | memory.soft_limit_in_bytes | -1 MiB | rm_memory_soft_limit | true |
Stacks Collection
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Stacks Collection Data Retention | The amount of stacks data that is retained. After the retention limit is reached, the oldest data is deleted. | stacks_collection_data_retention | 100 MiB | stacks_collection_data_retention | false |
| Stacks Collection Directory | The directory in which stacks logs are placed. If not set, stacks are logged into a stacks subdirectory of the role's log directory. | stacks_collection_directory | stacks_collection_directory | false | |
| Stacks Collection Enabled | Whether or not periodic stacks collection is enabled. | stacks_collection_enabled | false | stacks_collection_enabled | true |
| Stacks Collection Frequency | The frequency with which stacks are collected. | stacks_collection_frequency | 5.0 second(s) | stacks_collection_frequency | false |
| Stacks Collection Method | The method used to collect stacks. The jstack option involves periodically running the jstack command against the role's daemon process. The servlet method is available for those roles that have an HTTP server endpoint exposing the current stacks traces of all threads. When the servlet method is selected, that HTTP endpoint is periodically scraped. | stacks_collection_method | jstack | stacks_collection_method | false |
regionserverdefaultgroup
Advanced
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| RegionServer Environment Advanced Configuration Snippet (Safety Valve) | For advanced use only, key-value pairs (one on each line) to be inserted into a role's environment. Applies to configurations of this role except client configuration. | REGIONSERVER_role_env_safety_valve | false | ||
| Hadoop Metrics Advanced Configuration Snippet (Safety Valve) | Advanced Configuration Snippet (Safety Valve) for Hadoop Metrics. Properties will be inserted into hadoop-metrics.properties for this role only. Note that Cloudera Manager tunes hadoop-metrics.properties to work optimally with its Service Monitoring features. By overriding the default, Cloudera Manager might not be able to provide accurate monitoring information, health tests or alerts. | hadoop_metrics_safety_valve | false | ||
| HBase Coprocessor Region Classes | List of coprocessors that are loaded by default on all tables. For any override coprocessor method, these classes will be called in order. After implementing your own coprocessor, just put it in HBase's classpath and add the fully qualified class name here. A coprocessor can also be loaded on demand by setting HTableDescriptor. | hbase.coprocessor.region.classes | hbase_coprocessor_region_classes | false | |
| RegionServer Advanced Configuration Snippet (Safety Valve) for hbase-site.xml | For advanced use only, a string to be inserted into hbase-site.xml for this role only. | hbase_regionserver_config_safety_valve | false | ||
| Java Configuration Options for HBase RegionServer | These arguments will be passed as part of the Java command line. Commonly, garbage collection flags or extra debugging flags would be passed here. | -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:-CMSConcurrentMTEnabled -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled | hbase_regionserver_java_opts | false | |
| RegionServer Logging Advanced Configuration Snippet (Safety Valve) | For advanced use only, a string to be inserted into log4j.properties for this role only. | log4j_safety_valve | false | ||
| Heap Dump Directory | Path to directory where heap dumps are generated when java.lang.OutOfMemoryError error is thrown. This directory is automatically created if it does not exist. If this directory already exists, role user must have write access to this directory. If this directory is shared among multiple roles, it should have 1777 permissions. The heap dump files are created with 600 permissions and are owned by the role user. The amount of free space in this directory should be greater than the maximum Java Process heap size configured for this role. | oom_heap_dump_dir | /tmp | oom_heap_dump_dir | false |
| Dump Heap When Out of Memory | When set, generates heap dump file when java.lang.OutOfMemoryError is thrown. | false | oom_heap_dump_enabled | true | |
| Kill When Out of Memory | When set, a SIGKILL signal is sent to the role process when java.lang.OutOfMemoryError is thrown. | true | oom_sigkill_enabled | true | |
| Automatically Restart Process | When set, this role's process is automatically (and transparently) restarted in the event of an unexpected failure. | false | process_auto_restart | true |
Logs
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| RegionServer Log Directory | Directory where RegionServer will place its log files. | hadoop.log.dir | /var/log/hbase | hbase_regionserver_log_dir | false |
| RegionServer Logging Threshold | The minimum log level for RegionServer logs | INFO | log_threshold | false | |
| RegionServer Maximum Log File Backups | The maximum number of rolled log files to keep for RegionServer logs. Typically used by log4j or logback. | 10 | max_log_backup_index | false | |
| RegionServer Max Log Size | The maximum size, in megabytes, per log file for RegionServer logs. Typically used by log4j or logback. | 200 MiB | max_log_size | false |
Metrics
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Hadoop Metrics Class | Implementation daemons will use to report some internal statistics. The default (NoEmitMetricsContext) will display metrics on /metrics on the status port. The GangliaContext and GangliaContext31 classes will report metrics to your specified Ganglia Monitoring Daemons (gmond). The ganglia wire format changed incompatibly at version 3.1.0. If you are running any version of ganglia 3.1.0 or newer, use the GangliaContext31 metric class; otherwise, use the GangliaContext metric class. | org.apache.hadoop.metrics.spi.NoEmitMetricsContext | hadoop_metrics_class | false | |
| Hadoop Metrics Output Directory | If using FileContext, directory to write metrics to. | /tmp/metrics | hadoop_metrics_dir | false | |
| Hadoop Metrics Ganglia Servers | If using GangliaContext, a comma-delimited list of host:port pairs pointing to 'gmond' servers you would like to publish metrics to. In practice, this set of 'gmond' should match the set of 'gmond' in your 'gmetad' datasource list for the cluster. | hadoop_metrics_ganglia_servers | false | ||
| Extended Period | Time period in seconds to reset long-running metrics (e.g. compactions). This is an HBase specific configuration. | hbase.extendedperiod | 1 hour(s) | hbase_metrics_extended_period | false |
Monitoring
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Enable Health Alerts for this Role | When set, Cloudera Manager will send alerts when the health of this role reaches the threshold specified by the EventServer setting eventserver_health_events_alert_threshold | true | enable_alerts | false | |
| Enable Configuration Change Alerts | When set, Cloudera Manager will send alerts when this entity's configuration changes. | false | enable_config_alerts | false | |
| Heap Dump Directory Free Space Monitoring Absolute Thresholds | The health test thresholds for monitoring of free space on the filesystem that contains this role's heap dump directory. | Warning: 10 GiB, Critical: 5 GiB | heap_dump_directory_free_space_absolute_thresholds | false | |
| Heap Dump Directory Free Space Monitoring Percentage Thresholds | The health test thresholds for monitoring of free space on the filesystem that contains this role's heap dump directory. Specified as a percentage of the capacity on that filesystem. This setting is not used if a Heap Dump Directory Free Space Monitoring Absolute Thresholds setting is configured. | Warning: Never, Critical: Never | heap_dump_directory_free_space_percentage_thresholds | false | |
| Log Directory Free Space Monitoring Absolute Thresholds | The health test thresholds for monitoring of free space on the filesystem that contains this role's log directory. | Warning: 10 GiB, Critical: 5 GiB | log_directory_free_space_absolute_thresholds | false | |
| Log Directory Free Space Monitoring Percentage Thresholds | The health test thresholds for monitoring of free space on the filesystem that contains this role's log directory. Specified as a percentage of the capacity on that filesystem. This setting is not used if a Log Directory Free Space Monitoring Absolute Thresholds setting is configured. | Warning: Never, Critical: Never | log_directory_free_space_percentage_thresholds | false | |
| Rules to Extract Events from Log Files | This file contains the rules which govern how log messages are turned into events by the custom log4j appender that this role loads.
It is in JSON format, and is composed of a list of rules. Every log message is evaluated against each of these rules in turn to decide whether or not to send an event for that message. Each rule has
some or all of the following fields:
|
version: 0, rules: [ alert: false, rate: 1, periodminutes: 1, threshold:FATAL, alert: false, rate: 0, threshold:WARN, content: .* is deprecated. Instead, use .*, alert: false, rate: 0, threshold:WARN, content: .* is deprecated. Use .* instead, alert: false, rate: 0, exceptiontype: java.io.IOException, alert: false, rate: 0, exceptiontype: java.net.SocketException, alert: false, rate: 0, exceptiontype: java.net.SocketClosedException, alert: false, rate: 0, exceptiontype: java.io.EOFException, alert: false, rate: 0, exceptiontype: java.nio.channels.CancelledKeyException, alert: false, rate: 0, threshold:WARN, content: IPC Server handler.*took.*appending an edit to hlog.*, alert: false, rate: 1, periodminutes: 1, threshold:WARN, content: ABORTING region server serverName.*, alert: false, rate: 1, periodminutes: 1, threshold:WARN, content: DFSOutputStream ResponseProcessor exception.*, alert: false, rate: 1, periodminutes: 1, threshold:WARN, content: Error Recovery for block blk.*, alert: false, rate: 1, periodminutes: 1, threshold:WARN, content: Failed init, alert: false, rate: 1, periodminutes: 1, threshold:WARN, content: Problem renewing lease for DFSClient.*, alert: false, rate: 1, periodminutes: 1, threshold:WARN, content: remote error telling master we are up, alert: false, rate: 1, periodminutes: 1, threshold:WARN, content: Session.*for server.*closing socket connection and attempting reconnect., alert: false, rate: 1, periodminutes: 2, exceptiontype: .*, alert: false, rate: 0, threshold:WARN, content:Error executing shell command .+ No such process.+, alert: false, rate: 0, threshold:WARN, content:.*attempt to override final parameter.+, alert: false, rate: 0, threshold:WARN, content:[^ ]+ is a deprecated filesystem name. Use.*, alert: false, rate: -1, threshold:INFO, content: ^Starting .*compaction on region (.+)$, attribute:CATEGORY: HBASE, attribute:EVENTCODE: EV_HBASE_COMPACTION_REGION_STARTED, attribute:SEVERITY: INFORMATIONAL, group0: REGION, alert: false, rate: -1, threshold:INFO, content: ^completed compaction on region (.+) after (.+)$, attribute:CATEGORY: HBASE, attribute:EVENTCODE: EV_HBASE_COMPACTION_REGION_COMPLETED, attribute:SEVERITY: INFORMATIONAL, group0: REGION, group1: DURATION, alert: false, rate: -1, threshold:INFO, content: ^Starting compaction on (.+) in region (.+)$, attribute:CATEGORY: HBASE, attribute:EVENTCODE: EV_HBASE_COMPACTION_COLUMN_FAMILY_STARTED, attribute:SEVERITY: INFORMATIONAL, group0: COLUMN_FAMILY, group1: REGION, alert: false, rate: -1, threshold:INFO, content: ^completed compaction: regionName=(.+), storeName=(.+), fileCount=(.+), fileSize=(.+), priority=(.+), time=(.+); duration=(.+)$, attribute:CATEGORY: HBASE, attribute:EVENTCODE: EV_HBASE_COMPACTION_COLUMN_FAMILY_COMPLETED, attribute:SEVERITY: INFORMATIONAL, group0: REGION, group1: COLUMN_FAMILY, group2: FILE_COUNT, group3: FILE_SIZE, group4: PRIORITY, group5: REQUEST_TIME_NANOS, group6: DURATION, alert: false, rate: -1, threshold:INFO, content: ^Completed compaction: Request = regionName=(.+), storeName=(.+), fileCount=(.+), fileSize=(.+), priority=(.+), time=(.+); duration=(.+)$, attribute:CATEGORY: HBASE, attribute:EVENTCODE: EV_HBASE_COMPACTION_COLUMN_FAMILY_COMPLETED, attribute:SEVERITY: INFORMATIONAL, group0: REGION, group1: COLUMN_FAMILY, group2: FILE_COUNT, group3: FILE_SIZE, group4: PRIORITY, group5: REQUEST_TIME_NANOS, group6: DURATION, alert: false, rate: -1, threshold:INFO, content: ^aborted compaction: regionName=(.+), storeName=(.+), fileCount=(.+), fileSize=(.+), priority=(.+), time=(.+); duration=(.+)$, attribute:CATEGORY: HBASE, attribute:EVENTCODE: EV_HBASE_COMPACTION_COLUMN_FAMILY_ABORTED, attribute:SEVERITY: IMPORTANT, group0: REGION, group1: COLUMN_FAMILY, group2: FILE_COUNT, group3: FILE_SIZE, group4: PRIORITY, group5: REQUEST_TIME_NANOS, group6: DURATION, alert: false, rate: -1, threshold:INFO, content: ^Finished memstore flush of .+ for region (.+) in (.+), sequenceid=(.+), compaction requested=(.+)$, attribute:CATEGORY: HBASE, attribute:EVENTCODE: EV_HBASE_FLUSH_COMPLETED, attribute:SEVERITY: INFORMATIONAL, group0: REGION, group1: DURATION, group2: SEQUENCE_ID, group3: COMPACTION_REQUESTED , alert: false, rate: -1, threshold:INFO, content: ^Flush of region (.+) due to global heap pressure$, attribute:CATEGORY: HBASE, attribute:EVENTCODE: EV_HBASE_FLUSH_DUE_TO_HEAP_PRESSURE, attribute:SEVERITY: IMPORTANT, group0: REGION, alert: false, rate: -1, threshold:WARN, content: ^Region (.+) has too many store files; delaying flush up to 90000ms$, attribute:CATEGORY: HBASE, attribute:EVENTCODE: EV_HBASE_FLUSH_DELAYED_TOO_MANY_STORE_FILES, attribute:SEVERITY: CRITICAL, group0: REGION, alert: false, rate: -1, threshold:INFO, content: ^Starting split of region (.+)$, attribute:CATEGORY: HBASE, attribute:EVENTCODE: EV_HBASE_SPLIT_STARTED, attribute:SEVERITY: INFORMATIONAL, group0: REGION, alert: false, rate: -1, threshold:INFO, content: ^Running rollback/cleanup of failed split of (.+);.+$, attribute:CATEGORY: HBASE, attribute:EVENTCODE: EV_HBASE_SPLIT_ABORTED, attribute:SEVERITY: IMPORTANT, group0: REGION, alert: false, rate: -1, threshold:INFO, content: ^Region split, hbase:meta updated, and report to master. Parent=(.+), new regions: (.+, .*, .+), (.+, .*, .+). Split took (.+)$, attribute:CATEGORY: HBASE, attribute:EVENTCODE: EV_HBASE_SPLIT_COMPLETED, attribute:SEVERITY: INFORMATIONAL, group0: REGION, group1: DAUGHTER_REGIONS, group2: DAUGHTER_REGIONS, group3: DURATION, alert: false, rate: -1, threshold:INFO, content: ^Region split, META updated, and report to master. Parent=(.+), new regions: (.+, .*, .+), (.+, .*, .+). Split took (.+)$, attribute:CATEGORY: HBASE, attribute:EVENTCODE: EV_HBASE_SPLIT_COMPLETED, attribute:SEVERITY: INFORMATIONAL, group0: REGION, group1: DAUGHTER_REGIONS, group2: DAUGHTER_REGIONS, group3: DURATION, alert: false, rate: 1, periodminutes: 1, threshold:WARN ] | log_event_whitelist | false | |
| Process Swap Memory Thresholds | The health test thresholds on the swap memory usage of the process. | Warning: Any, Critical: Never | process_swap_memory_thresholds | false | |
| HBase RegionServer Compaction Queue Monitoring Thresholds | The health test thresholds of the weighted average size of the HBase RegionServer compaction queue over a recent period. See HBase RegionServer Compaction Queue Monitoring Period. | Warning: 10.0, Critical: Never | regionserver_compaction_queue_thresholds | false | |
| HBase RegionServer Compaction Queue Monitoring Period | The period over which to compute the moving average of the compaction queue size. | 5 minute(s) | regionserver_compaction_queue_window | false | |
| HBase Region Server Connectivity Tolerance at Startup | The amount of time to wait for the HBase Region Server to fully start up and connect to the HBase Master before enforcing the connectivity check. | 3 minute(s) | regionserver_connectivity_tolerance | false | |
| File Descriptor Monitoring Thresholds | The health test thresholds of the number of file descriptors used. Specified as a percentage of file descriptor limit. | Warning: 50.0 %, Critical: 70.0 % | regionserver_fd_thresholds | false | |
| HBase RegionServer Flush Queue Monitoring Thresholds | The health test thresholds of the average size of the HBase RegionServer flush queue over a recent period. See HBase RegionServer Flush Queue Monitoring Period. | Warning: 10.0, Critical: Never | regionserver_flush_queue_thresholds | false | |
| HBase RegionServer Flush Queue Monitoring Period | The period over which to compute the moving average of the flush queue size. | 5 minute(s) | regionserver_flush_queue_window | false | |
| Garbage Collection Duration Thresholds | The health test thresholds for the weighted average time spent in Java garbage collection. Specified as a percentage of elapsed wall clock time. | Warning: 30.0, Critical: 60.0 | regionserver_gc_duration_thresholds | false | |
| Garbage Collection Duration Monitoring Period | The period to review when computing the moving average of garbage collection time. | 5 minute(s) | regionserver_gc_duration_window | false | |
| RegionServer Host Health Test | When computing the overall RegionServer health, consider the host's health. | true | regionserver_host_health_enabled | false | |
| HBase RegionServer to Master Connectivity Test | Enables the health test that the RegionServer is connected to the Master | true | regionserver_master_connectivity_enabled | false | |
| HBase RegionServer Memstore Size Thresholds | The health test thresholds of the total size of RegionServer's memstores. Specified as a percentage of the configured upper limit. See Maximum Size of All Memstores in RegionServer. | Warning: 95.0 %, Critical: 100.0 % | regionserver_memstore_size_thresholds | false | |
| HBase RegionServer HDFS Read Latency Thresholds | The health test thresholds of the latency that the RegionServer sees for HDFS read operations | Warning: 50 millisecond(s), Critical: 100 millisecond(s) | regionserver_read_latency_thresholds | false | |
| HBase RegionServer HDFS Read Latency Monitoring Period | The period over which to compute the moving average of the HDFS read latency of the HBase RegionServer. | 5 minute(s) | regionserver_read_latency_window | false | |
| RegionServer Process Health Test | Enables the health test that the RegionServer's process state is consistent with the role configuration | true | regionserver_scm_health_enabled | false | |
| Percentage of Heap Used by HStoreFile Index | The health test thresholds of the size used by the HStoreFile index. Specified as a percentage of the total heap size. | Warning: 10.0 %, Critical: Never | regionserver_store_file_idx_size_thresholds | false | |
| HBase RegionServer HDFS Sync Latency Thresholds | The health test thresholds for the latency of HDFS write operations that the RegionServer detects | Warning: 500 millisecond(s), Critical: 5 second(s) | regionserver_sync_latency_thresholds | false | |
| HBase RegionServer HDFS Sync Latency Monitoring Period | The period over which to compute the moving average of the HDFS sync latency of the HBase RegionServer. | 5 minute(s) | regionserver_sync_latency_window | false | |
| Web Metric Collection | Enables the health test that the Cloudera Manager Agent can successfully contact and gather metrics from the web server. | true | regionserver_web_metric_collection_enabled | false | |
| Web Metric Collection Duration | The health test thresholds on the duration of the metrics request to the web server. | Warning: 10 second(s), Critical: Never | regionserver_web_metric_collection_thresholds | false | |
| Role Triggers | The configured triggers for this role. This is a JSON formatted list of triggers. These triggers are evaluated as part as the health
system. Every trigger expression is parsed, and if the trigger condition is met, the list of actions provided in the trigger expression is executed. Each trigger has all of the following fields:
|
[] | role_triggers | true | |
| Unexpected Exits Thresholds | The health test thresholds for unexpected exits encountered within a recent period specified by the unexpected_exits_window configuration for the role. | Warning: Never, Critical: Any | unexpected_exits_thresholds | false | |
| Unexpected Exits Monitoring Period | The period to review when computing unexpected exits. | 5 minute(s) | unexpected_exits_window | false |
Other
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| HBase Hash Type | The hashing algorithm for use in HashFunction. Two values are supported: 'murmur' (for MurmurHash) and 'jenkins' (for JenkinsHash). | hbase.hash.type | murmur | hbase_hash_type | false |
| HBase HRegion Major Compaction | The time, in milliseconds, between 'major' compactions of all HStoreFiles in a region. To disable automated major compactions, set this value to 0. | hbase.hregion.majorcompaction | 1 day(s) | hbase_hregion_majorcompaction | false |
| HBase Maximum File Size | Maximum HStoreFile size. If any one of a column families' HStoreFiles has grown to exceed this value, the hosting HRegion is split in two. | hbase.hregion.max.filesize | 10 GiB | hbase_hregion_max_filesize | false |
| HBase Memstore Block Multiplier | Blocks writes if the size of the memstore increases to the value of 'hbase.hregion.block.memstore' multiplied by the value of 'hbase.hregion.flush.size' bytes. This setting is useful for preventing runaway memstore during spikes in update traffic. Without an upper-bound, memstore fills such that when it flushes, the resultant process of flushing files take a long time to compact or split, or worse, an "out of memory" error occurs. | hbase.hregion.memstore.block.multiplier | 2 | hbase_hregion_memstore_block_multiplier | false |
| HBase Memstore Flush Size | Memstore will be flushed to disk if size of the memstore exceeds this value in number of bytes. This value is checked by a thread that runs the frequency specified by hbase.server.thread.wakefrequency. | hbase.hregion.memstore.flush.size | 128 MiB | hbase_hregion_memstore_flush_size | false |
| Chunk Size Allocated by MSLAB Allocation Scheme | The size of the chunks allocated by MSLAB, in bytes. | hbase.hregion.memstore.mslab.chunksize | 2 MiB | hbase_hregion_memstore_mslab_chunksize | false |
| Enable MSLAB Allocation Scheme | Enable MemStore-Local Allocation Buffer (MSLAB) Allocation Scheme. Note: This feature is experimental in CDH3. | hbase.hregion.memstore.mslab.enabled | true | hbase_hregion_memstore_mslab_enabled | false |
| Maximum Byte Array from MSLAB Allocation Scheme | The maximum size byte array that should come from the MSLAB, in bytes. | hbase.hregion.memstore.mslab.max.allocation | 256 KiB | hbase_hregion_memstore_mslab_max_allocation | false |
| HBase Memstore Pre-close Flush Size | If the memstores in a region are this size or larger when closing, run a pre-flush process to clear out memstores before putting up the region closed flag and taking the region offline. On close, a flush process is run under the close flag up to empty memory. During this time, the region is offline and no writes are taken. If the memstore content is large, the flush process could take a long time to complete. The pre-flush process cleans out the bulk of the memstore before putting up the close flag and taking the region offline, so that the flush process that runs under the close flag has little to do. | hbase.hregion.preclose.flush.size | 5 MiB | hbase_hregion_preclose_flush_size | false |
| HStore Blocking Store Files | If there are more than this number of HStoreFiles in any one HStore, then updates are blocked for this HRegion until a compaction is completed, or until the value specified for 'hbase.hstore.blockingWaitTime' has been exceeded. | hbase.hstore.blockingStoreFiles | 7 | hbase_hstore_blockingStoreFiles | false |
| HStore Blocking Wait Time | The period of time that an HRegion will block updates after reaching the HStoreFile limit that is specified by 'hbase.hstore.blockingStoreFiles'. After this time has elapsed, the HRegion will stop blocking updates even if a compaction has not been completed. | hbase.hstore.blockingWaitTime | 1 minute(s), 30 second(s) | hbase_hstore_blockingWaitTime | false |
| HStore Compaction Threshold | If this number of HStoreFiles in any one HStore is exceeded, then a compaction is run to rewrite all HStoreFiles files as one HStoreFile. (One HStoreFile is written per flush of memstore.) You can delay compaction by specifying a larger number, but the compaction will take longer when it does run. During a compaction, updates cannot be flushed to disk. Long compactions require memory sufficient to carry the logging of all updates across the duration of the compaction. If too large, clients timeout during compaction. | hbase.hstore.compactionThreshold | 3 | hbase_hstore_compactionThreshold | false |
| Maximum Number of HStoreFiles Compaction | Maximum number of HStoreFiles to compact per minor compaction. | hbase.hstore.compaction.max | hbase_hstore_compaction_max | false | |
| HBase RegionServer Interface Class | An interface that is assignable to HRegionInterface. Used in HBase Client for opening a proxy to a remote HBase RegionServer. | hbase.regionserver.class | hbase_regionserver_class | false | |
| RegionServer Codecs | Comma-separated list of codecs that the RegionServer requires to start. Use this setting to make sure that all RegionServers joining a cluster are installed with a particular set of codecs. | hbase.regionserver.codecs | hbase_regionserver_codecs | false | |
| Low Watermark for Memstore Flush | When memstores are being forced to flush to make room in memory, keep flushing until this amount is reached. If this amount is equal to 'hbase.regionserver.global.memstore.upperLimit', then minimum possible flushing will occur when updates are blocked due to memstore limiting. | hbase.regionserver.global.memstore.lowerLimit | 0.35 | hbase_regionserver_global_memstore_lowerLimit | false |
| Maximum Size of All Memstores in RegionServer | Maximum size of all memstores in a RegionServer before new updates are blocked and flushes are forced. | hbase.regionserver.global.memstore.upperLimit | 0.4 | hbase_regionserver_global_memstore_upperLimit | false |
| HBase RegionServer Handler Count | Number of RPC Server instances spun up on RegionServers. | hbase.regionserver.handler.count | 10 | hbase_regionserver_handler_count | false |
| HLog Reader Implementation | The HLog file reader implementation. | hbase.regionserver.hlog.reader.impl | hbase_regionserver_hlog_reader_impl | false | |
| HLog Writer Implementation | The HLog file writer implementation. | hbase.regionserver.hlog.writer.impl | hbase_regionserver_hlog_writer_impl | false | |
| HBase RegionServer Lease Period | The lease period, in milliseconds, for the HBase RegionServer. Clients must report in within this period or else they are considered dead. | hbase.regionserver.lease.period | 1 minute(s) | hbase_regionserver_lease_period | false |
| HBase RegionServer Log Roll Period | Period, in milliseconds, at which to roll the commit log. | hbase.regionserver.logroll.period | 1 hour(s) | hbase_regionserver_logroll_period | false |
| Maximum number of Write-Ahead Log (WAL) files | Maximum number of Write-Ahead Log (WAL) files. This value multiplied by HDFS Block Size (dfs.blocksize) is the size of the WAL that will need to be replayed when a server crashes. This value is inversely proportional to the frequency of flushes to disk. | hbase.regionserver.maxlogs | 32 | hbase_regionserver_maxlogs | false |
| HBase RegionServer Meta-Handler Count | Number of handlers for processing priority requests in a RegionServer. | hbase.regionserver.metahandler.count | 10 | hbase_regionserver_metahandler_count | false |
| HBase RegionServer Message Interval | Interval, in milliseconds, between messages from the RegionServer to the HBase Master. Use a high value such as 3000 for clusters that have more than 10 hosts. | hbase.regionserver.msginterval | 3 second(s) | hbase_regionserver_msginterval | false |
| RegionServer Reservation Blocks | The number of reservation blocks that are used to prevent unstable RegionServers caused by an OOME. | hbase.regionserver.nbreservationblocks | 4 | hbase_regionserver_nbreservationblocks | false |
| Synch Interval of HLog Entries | Sync the HLog to HDFS after this interval, in milliseconds, if it has not accumulated the number of HLog Entries specified to trigger a sync. | hbase.regionserver.optionallogflushinterval | 1 second(s) | hbase_regionserver_optionallogflushinterval | false |
| HBase Region Split Limit | Limit for the number of regions after which no more region splitting should take place. This is not a hard limit for the number of regions but acts as a guideline for the RegionServer to stop splitting after a certain limit. | hbase.regionserver.regionSplitLimit | 2147483647 | hbase_regionserver_regionSplitLimit | false |
| HFile Block Cache Size | Percentage of maximum heap (-Xmx setting) to allocate to block cache used by HFile/StoreFile. To disable, set this value to 0 . | hfile.block.cache.size | 0.25 | hfile_block_cache_size | false |
Performance
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Maximum Process File Descriptors | If configured, overrides the process soft and hard rlimits (also called ulimits) for file descriptors to the configured value. | 32768 | rlimit_fds | false |
Ports and Addresses
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| HBase RegionServer Bind to Wildcard Address | When true, HBase RegionServer will bind to 0.0.0.0. Only available in CDH 4.3 and later. | hbase.regionserver.ipc.address | true | hbase_regionserver_bind_to_wildcard_address | true |
| RegionServer DNS Network Interface | The name of the DNS Network Interface from which a RegionServer should report its IP address. | hbase.regionserver.dns.interface | hbase_regionserver_dns_interface | false | |
| RegionServer DNS Nameserver | The host name or IP address of the DNS name server which a RegionServer should use to determine the host name used by the HBase Master for communication and display purposes. | hbase.regionserver.dns.nameserver | hbase_regionserver_dns_nameserver | false | |
| HBase RegionServer Web UI Address | The address for the HBase RegionServer web UI | hbase.regionserver.info.bindAddress | hbase_regionserver_info_bindAddress | false | |
| HBase RegionServer Web UI port | The port for the HBase RegionServer web UI. Set to -1 to disable RegionServer web UI. | hbase.regionserver.info.port | 60030 | hbase_regionserver_info_port | false |
| HBase RegionServer Port | The port that an HBase RegionServer binds to. | hbase.regionserver.port | 60020 | hbase_regionserver_port | false |
Resource Management
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Java Heap Size of HBase RegionServer in Bytes | Maximum size in bytes for the Java Process heap memory. Passed to Java -Xmx. | 4 GiB | hbase_regionserver_java_heapsize | false | |
| Cgroup CPU Shares | Number of CPU shares to assign to this role. The greater the number of shares, the larger the share of the host's CPUs that will be given to this role when the host experiences CPU contention. Must be between 2 and 262144. Defaults to 1024 for processes not managed by Cloudera Manager. | cpu.shares | 1024 | rm_cpu_shares | true |
| Cgroup I/O Weight | Weight for the read I/O requests issued by this role. The greater the weight, the higher the priority of the requests when the host experiences I/O contention. Must be between 100 and 1000. Defaults to 1000 for processes not managed by Cloudera Manager. | blkio.weight | 500 | rm_io_weight | true |
| Cgroup Memory Hard Limit | Hard memory limit to assign to this role, enforced by the Linux kernel. When the limit is reached, the kernel will reclaim pages charged to the process. If reclaiming fails, the kernel may kill the process. Both anonymous as well as page cache pages contribute to the limit. Use a value of -1 B to specify no limit. By default processes not managed by Cloudera Manager will have no limit. | memory.limit_in_bytes | -1 MiB | rm_memory_hard_limit | true |
| Cgroup Memory Soft Limit | Soft memory limit to assign to this role, enforced by the Linux kernel. When the limit is reached, the kernel will reclaim pages charged to the process if and only if the host is facing memory pressure. If reclaiming fails, the kernel may kill the process. Both anonymous as well as page cache pages contribute to the limit. Use a value of -1 B to specify no limit. By default processes not managed by Cloudera Manager will have no limit. | memory.soft_limit_in_bytes | -1 MiB | rm_memory_soft_limit | true |
Security
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| HBase Secure Bulk Load Directory | The directory for HBase secure bulk Load. | hbase.bulkload.staging.dir | /tmp/hbase-staging | hbase_bulkload_staging_dir | false |
Stacks Collection
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Stacks Collection Data Retention | The amount of stacks data that is retained. After the retention limit is reached, the oldest data is deleted. | stacks_collection_data_retention | 100 MiB | stacks_collection_data_retention | false |
| Stacks Collection Directory | The directory in which stacks logs are placed. If not set, stacks are logged into a stacks subdirectory of the role's log directory. | stacks_collection_directory | stacks_collection_directory | false | |
| Stacks Collection Enabled | Whether or not periodic stacks collection is enabled. | stacks_collection_enabled | false | stacks_collection_enabled | true |
| Stacks Collection Frequency | The frequency with which stacks are collected. | stacks_collection_frequency | 5.0 second(s) | stacks_collection_frequency | false |
| Stacks Collection Method | The method used to collect stacks. The jstack option involves periodically running the jstack command against the role's daemon process. The servlet method is available for those roles that have an HTTP server endpoint exposing the current stacks traces of all threads. When the servlet method is selected, that HTTP endpoint is periodically scraped. | stacks_collection_method | jstack | stacks_collection_method | false |
service_wide
Advanced
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| HBase Coprocessor Abort on Error | Set to true to cause the hosting server (Master or RegionServer) to abort if a coprocessor throws a Throwable object that is not IOException or a subclass of IOException. Setting it to true might be useful in development environments where one wants to terminate the server as soon as possible to simplify coprocessor failure analysis. | hbase.coprocessor.abortonerror | false | hbase_coprocessor_abort_on_error | false |
| HBase Service Advanced Configuration Snippet (Safety Valve) for core-site.xml | For advanced use only, a string to be inserted into core-site.xml. Applies to configurations of all roles in this service except client configuration. | hbase_core_site_safety_valve | false | ||
| HBase Service Advanced Configuration Snippet (Safety Valve) for hbase-site.xml | For advanced use only, a string to be inserted into hbase-site.xml. Applies to configurations of all roles in this service except client configuration. | hbase_service_config_safety_valve | false | ||
| HBase Service Environment Advanced Configuration Snippet (Safety Valve) | For advanced use only, key-value pairs (one on each line) to be inserted into a role's environment. Applies to configurations of all roles in this service except client configuration. | hbase_service_env_safety_valve | false | ||
| HBase User to Impersonate | The user the management services impersonate when connecting to HBase. If no value is specified, the HBase superuser is used. | hbase_user_to_impersonate | false | ||
| HBASE Client Advanced Configuration Snippet (Safety Valve) for navigator.client.properties | For advanced use only, a string to be inserted into the client configuration for navigator.client.properties. | navigator_client_config_safety_valve | false | ||
| System Group | The group that this service's processes should run as. | hbase | process_groupname | true | |
| System User | The user that this service's processes should run as. | hbase | process_username | true |
Backup
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Enable Indexing | Allow indexing of tables in HBase by Lily HBase Indexer. Note: Replication must be enabled for indexing to work. | false | hbase_enable_indexing | false | |
| Enable Replication | Allow HBase tables to be replicated. | hbase.replication | false | hbase_enable_replication | false |
| Replication Batch Size | Maximum number of hlog entries to replicate in one go. If this is large, and a consumer takes a while to process the events, the HBase RPC call will time out. | replication.source.nb.capacity | 1000 | hbase_replication_source_nb_capacity | false |
| Replication Source Ratio | Ratio of Lily HBase Indexers used by each HBase RegionServer while doing replication. | replication.source.ratio | 1.0 | hbase_replication_source_ratio | false |
| Enable Snapshots | Enable snapshots. Disabling snapshots requires deletion of all snapshots before restarting the HBase master; the HBase master will not start if snapshots are disabled and snapshots exist. | hbase.snapshot.enabled | true | hbase_snapshot_enabled | false |
| HBase Master Snapshot Waiting Timeout | The timeout before injecting a snapshot timeout error when waiting for a snapshot completion. | hbase.snapshot.master.timeoutMillis | 1 minute(s) | hbase_snapshot_master_timeoutMillis | false |
| HBase Master Snapshot Timeout | The maximum amount of time the HBase master waits for a snapshot to complete. | hbase.snapshot.master.timeout.millis | 1 minute(s) | hbase_snapshot_master_timeout_millis | false |
| HBase RegionServer Snapshot Timeout | The maximum amount of time the Hbase RegionServer waits for a snapshot to complete. | hbase.snapshot.region.timeout | 1 minute(s) | hbase_snapshot_region_timeout | false |
| Amazon S3 Access Key ID for Remote Snapshots | Access key ID required to access Amazon S3 to store remote snapshots. | hbase_snapshot_s3_access_key_id | false | ||
| Amazon S3 Path for Remote Snapshots | Amazon S3 path where remote snapshots should be stored. | hbase_snapshot_s3_path | false | ||
| Scheduler Pool for Remote Snapshots in AWS S3 | Name of the scheduler pool to use for MR jobs created during export/import of remote snapshots in AWS S3. | hbase_snapshot_s3_scheduler_pool | false | ||
| AWS S3 Secret Access Key for Remote Snapshots | AWS secret access key required to access S3 to store remote snapshots. | hbase_snapshot_s3_secret_access_key | false |
Cloudera Navigator
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Enable Audit Collection | Enable collection of audit events from the service's roles. | true | navigator_audit_enabled | false | |
| Audit Event Filter | Event filters are defined in a JSON object like the following: { "defaultAction" : ("accept", "discard"),
"rules" : [ { "action" : ("accept", "discard"), "fields" : [ { "name" : "fieldName", "match" : "regex" } ] } ] } A filter has a default action and a list of rules, in order of precedence. Each
rule defines an action, and a list of fields to match against the audit event. A rule is "accepted" if all the listed field entries match the audit event. At that point, the action declared by the
rule is taken. If no rules match the event, the default action is taken. Actions default to "accept" if not defined in the JSON object. The following is the list of fields that can be filtered for
HBase events:
|
navigator.event.filter | comment : [ The default HBase audit event filter discards events that affect the , internal -ROOT-, .META. and _acl_ tables. ], defaultAction : accept, rules : [ action : discard, fields : [ name : tableName, match : (?:-ROOT-|.META.|_acl_|hbase:meta|hbase:acl) ] ] | navigator_audit_event_filter | false |
| Audit Queue Policy | Action to take when the audit event queue is full. Drop the event or shutdown the affected process. | navigator.batch.queue_policy | DROP | navigator_audit_queue_policy | false |
| Audit Event Tracker | Configures the rules for event tracking and coalescing. This feature is used to define equivalency between different audit events.
When events match, according to a set of configurable parameters, only one entry in the audit list is generated for all the matching events. Tracking works by keeping a reference to events when they
first appear, and comparing other incoming events against the "tracked" events according to the rules defined here. Event trackers are defined in a JSON object like the following: { "timeToLive" : [integer], "fields" : [ { "type" : [string], "name" : [string] } ] } Where:
|
navigator_event_tracker | comment : [ The default event tracker for HBase services defines equality by , comparing the username, operation, table name, family, and qualifier of , the events. ], timeToLive : 60000, fields : [ type: value, name : tableName , type: value, name : family , type: value, name : qualifier , type: value, name : operation , type: username, name : username ] | navigator_event_tracker | false |
Logs
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Audit Log Directory | Path to the directory where audit logs will be written. The directory will be created if it doesn't exist. | audit_event_log_dir | /var/log/hbase/audit | audit_event_log_dir | false |
| Maximum Audit Log File Size | Maximum size of audit log file in MB before it is rolled over. | navigator.audit_log_max_file_size | 100 MiB | navigator_audit_log_max_file_size | false |
| Number of Audit Logs to Retain | Maximum number of rolled over audit logs to retain. The logs will not be deleted if they contain audit events that have not yet been propagated to Audit Server. | navigator.client.max_num_audit_log | 10 | navigator_client_max_num_audit_log | false |
Monitoring
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Enable Log Event Capture | When set, each role identifies important log events and forwards them to Cloudera Manager. | true | catch_events | false | |
| Enable Service Level Health Alerts | When set, Cloudera Manager will send alerts when the health of this service reaches the threshold specified by the EventServer setting eventserver_health_events_alert_threshold | true | enable_alerts | false | |
| Enable Configuration Change Alerts | When set, Cloudera Manager will send alerts when this entity's configuration changes. | false | enable_config_alerts | false | |
| HBase Active Master Detection Window | The tolerance window that will be used in HBase service tests that depend on detection of the active HBase Master. | 3 minute(s) | hbase_active_master_detecton_window | false | |
| Backup Masters Health Test | When computing the overall HBase cluster health, consider the health of the backup HBase Masters. | true | hbase_backup_masters_health_enabled | false | |
| HBase Canary Unhealthy Region Count Alert Threshold | An alert is published if the HBase region health canary detects at least this many unhealthy regions. This setting takes precedence over the hbase_canary_alert_unhealthy_region_percent_threshold config. | hbase_canary_alert_unhealthy_region_count_threshold | false | ||
| HBase Canary Unhealthy Region Percentage Alert Threshold | An alert is published if the HBase region health canary detects at least this percentage of total regions are unhealthy. This threshold is used if the explicit count is not set via the hbase_canary_alert_unhealthy_region_count_threshold config. | 0.1 | hbase_canary_alert_unhealthy_region_percent_threshold | false | |
| HBase Hbck Alert Error Codes | An alert is published if the HBase Hbck tool finds any errors with matching codes. Possible error codes: UNKNOWN, NO_META_REGION, NULL_ROOT_REGION, NO_VERSION_FILE, NOT_IN_META_HDFS, NOT_IN_META, NOT_IN_META_OR_DEPLOYED, NOT_IN_HDFS_OR_DEPLOYED, NOT_IN_HDFS, SERVER_DOES_NOT_MATCH_META, NOT_DEPLOYED, MULTI_DEPLOYED, SHOULD_NOT_BE_DEPLOYED, MULTI_META_REGION, RS_CONNECT_FAILURE, FIRST_REGION_STARTKEY_NOT_EMPTY, LAST_REGION_ENDKEY_NOT_EMPTY, DUPE_STARTKEYS, HOLE_IN_REGION_CHAIN, OVERLAP_IN_REGION_CHAIN, REGION_CYCLE, DEGENERATE_REGION, ORPHAN_HDFS_REGION, LINGERING_SPLIT_PARENT, NO_TABLEINFO_FILE | NO_META_REGION, NULL_ROOT_REGION | hbase_hbck_alert_error_codes | false | |
| HBase Hbck Error Count Alert Threshold | An alert is published if the HBase Hbck tool detects at least this many errors across all tables in this service. Some errors are not associated with a region, e.g. 'RS_CONNECT_FAILURE'. If the value is not set, alerts will not be published based on the count of errors. | hbase_hbck_alert_error_count_threshold | false | ||
| HBase Hbck Region Error Count Alert Threshold | An alert is published if the HBase Hbck tool detects at least this many regions with errors across all tables in this service. If the value is not set, alerts will not be published based on the count of regions with errors. | hbase_hbck_alert_region_error_count_threshold | false | ||
| HBase Hbck Check HDFS | Configures whether the Hbck poller checks HDFS or not. Checking HBase tables and regions information on HDFS can take a while. | true | hbase_hbck_poller_check_hdfs | false | |
| HBase Hbck Poller | Enables the HBase Hbck Poller so that Hbck reports will be available. Enabling the Hbck poller will increase the amount of memory used by the Service Monitor. Consider increasing the Service Monitor Java heap size by an additional 3KB per region. For example, for a cluster with 10,000 regions, increase the JVM heap size by approximately 30MB. | false | hbase_hbck_poller_enabled | false | |
| HBase Hbck Poller Maximum Error Count | Maximum number of errors that the HBase Hbck poller will retain through a given run | 10000 | hbase_hbck_poller_max_error_count | false | |
| HBase Hbck Slow Run Alert Enabled | An alert is published if the HBase Hbck tool runs slowly. | true | hbase_hbck_slow_run_alert_enabled | false | |
| Active Master Health Test | When computing the overall HBase cluster health, consider the active HBase Master's health. | true | hbase_master_health_enabled | false | |
| HBase Region Health Canary | Enables the canary that checks HBase region availability by scanning a row from every region. | true | hbase_region_health_canary_enabled | false | |
| HBase Region Health Canary Exclude Tables | Tables to exclude in the HBase Region Health Canary which will scan a row from every region. | hbase_region_health_canary_exclude_tables | false | ||
| HBase Region Health Canary Slow Run Alert Enabled | An alert is published if the HBase region health canary runs slowly. | true | hbase_region_health_canary_slow_run_alert_enabled | false | |
| Healthy RegionServer Monitoring Thresholds | The health test thresholds of the overall RegionServer health. The check returns "Concerning" health if the percentage of "Healthy" RegionServers falls below the warning threshold. The check is unhealthy if the total percentage of "Healthy" and "Concerning" RegionServers falls below the critical threshold. | Warning: 95.0 %, Critical: 90.0 % | hbase_regionservers_healthy_thresholds | false | |
| Log Event Retry Frequency | The frequency in which the log4j event publication appender will retry sending undelivered log events to the Event server, in seconds | 30 | log_event_retry_frequency | false | |
| Service Triggers | The configured triggers for this service. This is a JSON formatted list of triggers. These triggers are evaluated as part as the
health system. Every trigger expression is parsed, and if the trigger condition is met, the list of actions provided in the trigger expression is executed. Each trigger has all of the following
fields:
|
[] | service_triggers | true | |
| Service Monitor Client Config Overrides | For advanced use only, a list of configuration properties that will be used by the Service Monitor instead of the current client configuration for the service. | <property><name>zookeeper.recovery.retry</name><value>0</value></property><property><name>zookeeper.recovery.retry.intervalmill</name><value>3000</value></property><property><name>hbase.zookeeper.recoverable.waittime</name><value>1000</value></property><property><name>zookeeper.session.timeout</name><value>30000</value></property><property><name>hbase.rpc.timeout</name><value>10000</value></property><property><name>hbase.client.retries.number</name><value>1</value></property><property><name>hbase.client.rpc.maxattempts</name><value>1</value></property><property><name>hbase.client.operation.timeout</name><value>10000</value></property> | smon_client_config_overrides | false | |
| Service Monitor Derived Configs Advanced Configuration Snippet (Safety Valve) | For advanced use only, a list of derived configuration properties that will be used by the Service Monitor instead of the default ones. | smon_derived_configs_safety_valve | false |
Other
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Maximum Size of HBase Client KeyValue | Specifies the combined maximum allowed size of a KeyValue instance. This option configures an upper boundary for a single entry saved in a storage file. This option prevents a region from splitting if the data is too large. Set this option to a fraction of the maximum region size. To disable this check, use a value of zero or less. | hbase.client.keyvalue.maxsize | 10 MiB | hbase_client_keyvalue_maxsize | false |
| HBase Client Pause | A general client pause time value. Used mostly as a time period to wait before retrying operations such as a failed get or region lookup. | hbase.client.pause | 1 second(s) | hbase_client_pause | false |
| Maximum HBase Client Retries | Maximum number of client retries. Used as a maximum for all operations such as fetching of the root region from the root RegionServer, getting a cell's value, and starting a row update. | hbase.client.retries.number | 10 | hbase_client_retries_number | false |
| HBase Client Scanner Caching | Number of rows to fetch when calling next on a scanner if it is not served from memory. Higher caching values enable faster scanners but require more memory and some calls of next may take longer when the cache is empty. | hbase.client.scanner.caching | 1 | hbase_client_scanner_caching | false |
| HBase Client Write Buffer | Write buffer size in bytes. A larger buffer requires more memory on both the client and the server because the server instantiates the passed write buffer to process it but reduces the number of remote procedure calls (RPC). To estimate the amount of server memory used, multiply the value of 'hbase.client.write.buffer' by the value of 'hbase.regionserver.handler.count'. | hbase.client.write.buffer | 2 MiB | hbase_client_write_buffer | false |
| Graceful Shutdown Timeout | Timeout for graceful shutdown of this HBase service. Once this timeout is reached, any remaining running roles are abruptly shutdown. A value of 0 means no timeout. | 3 minute(s) | hbase_graceful_stop_timeout | true | |
| RPC Timeout | Timeout for all HBase RPCs in milliseconds. | hbase.rpc.timeout | 1 minute(s) | hbase_rpc_timeout | false |
| HBase Server Thread Wake Frequency | Period of time, in milliseconds, to pause between searches for work. Used as a sleep interval by service threads such as a META scanner and log roller. | hbase.server.thread.wakefrequency | 10 second(s) | hbase_server_thread_wakefrequency | false |
| HBase Superusers | List of users or groups, who are allowed full privileges, regardless of stored ACLs, across the cluster. Only used when HBase security is enabled. | hbase.superuser | hbase_superuser | false | |
| HDFS Root Directory | The HDFS directory shared by HBase RegionServers. | hbase.rootdir | /hbase | hdfs_rootdir | true |
| HDFS Service | Name of the HDFS service that this HBase service instance depends on | hdfs_service | true | ||
| ZooKeeper Connection Retry Pause Duration | Period of time, in milliseconds, to pause between connection retries to ZooKeeper. Used together with ${zookeeper.retries} in an exponential backoff fashion when making queries to ZooKeeper. | zookeeper.pause | zookeeper_pause | false | |
| ZooKeeper Connection Retries | The number of times to retry connections to ZooKeeper. Used for reading and writing root region location. Used together with ${zookeeper.pause} in an exponential backoff fashion when making queries to ZooKeeper. | zookeeper.retries | zookeeper_retries | false | |
| ZooKeeper Service | Name of the ZooKeeper service that this HBase service instance depends on. | zookeeper_service | true | ||
| ZooKeeper Session Timeout | ZooKeeper session timeout in milliseconds. HBase passes this to the ZooKeeper quorum as the suggested maximum time for a session. See http://hadoop.apache.org/zookeeper/docs/current/zookeeperProgrammers.html#ch_zkSessions The client sends a requested timeout, the server responds with the timeout that it can give the client. | zookeeper.session.timeout | 60000 | zookeeper_session_timeout | false |
| ZooKeeper Znode Parent | The root znode for HBase in ZooKeeper. All of HBase's ZooKeeper files that are configured with a relative path will go under this node. By default, all of HBase's ZooKeeper file paths are configured with a relative path, so they will all go under this directory unless changed. | zookeeper.znode.parent | /hbase | zookeeper_znode_parent | true |
| ZooKeeper Znode Rootserver | Path to ZooKeeper Node holding root region location. This is written by the HBase Master and read by clients and RegionServers. If a relative path is given, the parent folder will be ${zookeeper.znode.parent}. By default, the root location is stored at /hbase/root-region-server. | zookeeper.znode.rootserver | root-region-server | zookeeper_znode_rootserver | true |
Performance
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| Enable HDFS Short Circuit Read | Enable HDFS short circuit read. This allows a client co-located with the DataNode to read HDFS file blocks directly. This gives a performance boost to distributed clients that are aware of locality. | dfs.client.read.shortcircuit | true | dfs_client_read_shortcircuit | false |
| HDFS Hedged Read Threadpool Size | Size of the threadpool used for hedged reads in hdfs clients. If a read from a block is slow, a parallel 'hedged' read will be started against a different block replica. The first one to return with a result is used while the other one is cancelled. This 'hedged' read feature helps rein in the outliers. A value of zero disables the feature. | dfs.client.hedged.read.threadpool.size | 0 | hbase_server_dfs_client_hedged_read_threadpool_size | false |
| HDFS Hedged Read Delay Threshold | Duration to wait before starting up a 'hedged' read. | dfs.client.hedged.read.threshold.millis | 500 millisecond(s) | hbase_server_dfs_client_hedged_read_threshold_millis | false |
| SplitLog Manager Timeout | Timeout (in ms) for the distributed log splitting manager to receive response from a worker. | hbase.splitlog.manager.timeout | 2 minute(s) | hbase_service_splitlog_manager_timeout | false |
Security
| Display Name | Description | Related Name | Default Value | API Name | Required |
|---|---|---|---|---|---|
| HBase Secure RPC Engine | Set to true to use HBase Secure RPC Engine for remote procedure calls (RPC). This is only effective in simple authentication mode. Does not provide authentication for RPC calls, but provides user information in the audit logs. Changing this setting requires a restart of this and all dependent services and redeployment of client configurations, along with a restart of the Service Monitor management role. | hbase.secure.rpc.engine | false | hbase_secure_rpc_engine | false |
| HBase Secure Authentication | Choose the authentication mechanism used by HBase. | hbase.security.authentication | simple | hbase_security_authentication | false |
| HBase Secure Authorization | Enable HBase authorization. | hbase.security.authorization | false | hbase_security_authorization | false |
| Kerberos Principal | Kerberos principal short name used by all roles of this service. | hbase | kerberos_princ_name | true |