Hive ACID metric properties for compaction observability

As an administrator, you must configure certain properties related to the Hive ACID metrics service to enable the collection of Hive ACID metrics that are required to display compaction-related alerts and charts in Cloudera Manager.

Basic properties

You must configure the following metrics to enable Hive ACID metrics collection:

hive.metastore.metrics.enabled: Default value: True; Enables or disables Hive metrics subsystem for the Hive Metastore (HMS) role.
hive.server2.metrics.enabled: Default value: True; Enables or disables Hive metrics subsystem for the HiveServer (HS2) role.
hive.metastore.acidmetrics.ext.on: Default value: True; Set this to property to True to collect additional acid related metrics outside of the Hive ACID metrics service. Enable this property for both HMS and HS2 roles.
note
Ensure that you have also enabled the hive.metastore.metrics.enabled property for HMS and the hive.server2.metrics.enabled property for HS2.
metastore.acidmetrics.thread.on: Default value: True; Specifies whether to run Hive ACID related metrics collection on this metastore instance. Enable this property in only one HMS instance and disable the property in all other instances.

Advanced properties

Set these additional properties to configure how Hive ACID metrics service collects metric data:

The following properties have to be configured in the HMS instance where metastore.acidmetrics.thread.on is enabled:

metastore.acidmetrics.check.interval: Default value: 300 seconds; Specifies the time (in seconds) between Hive ACID metric collection jobs.
metastore.acidmetrics.table.aborted.txns.threshold: Default value: 1500; Hive metrics subsystem collects the number of tables that have a large number of aborted transactions. This property specifies the minimum number of aborted transactions required to consider a table.
metastore.compactor.acid.metrics.logger.frequency: Default value: 360m; Specifies the logging frequency (in minutes) of Hive ACID metrics. Set this property in only one HMS instance and set the property to 0 in all other HMS instances to disable logging.

The following properties have to be configured in the HS2 (Hive on Tez) service:

hive.txn.acid.metrics.cache.size: Default value: 100; Specifies the size of the ACID metrics cache, which is the maximum number of partitioned and unpartitioned tables with the most deltas that is included in the list of active, obsolete, and small deltas. Valid values are between 0 to 500.
hive.txn.acid.metrics.cache.duration: Default value: 7200 seconds; Specifies the maximum lifetime (in seconds) for an entry in the ACID metrics cache.
hive.txn.acid.metrics.reporting.interval: Default value: 30 seconds; Specifies the reporting period (in seconds) for ACID metrics.
hive.txn.acid.metrics.delta.num.threshold: Default value: 100; Specifies the minimum number of active delta files that a table or partition must have to be included in the ACID metrics report.
hive.txn.acid.metrics.obsolete.delta.num.threshold: Default value: 100; Specifies the minimum number of obsolete delta files that a table or partition must have to be included in the ACID metrics report.
hive.txn.acid.metrics.delta.check.threshold: Default value: 300 seconds; Specifies the minimum age (in seconds) for delta files to be included in the ACID metrics report. Delta files less than this age are not included in the report.
hive.txn.acid.metrics.delta.pct.threshold: Default value: 0.01; Specifies the percentage size of the delta files relative to the base directory. Delta files whose size is smaller than this threshold value are considered as small deltas and are not included in the ACID metrics report. (1 = 100%, so the default 0.01 = 1%)