Several properties in the Hive and Hive metastore service configurations must be set
to enable automatic compaction. You need to check that the property settings are correct and
to add one of the properties to the Hive on Tez service. Automatic compaction will then
occur at regular intervals, but only if
necessary.
Initiator threads should run in only one Hive
Metastore server (even in high-availability / HA configurations). The HMS instances will
elect a single leader among themselves. There is no need to override the
hive.compactor.initiator.on
on the Hive metastore instance level.
For more information, see Hive Metastore leader election. Set the following
properties:
- In Hive metastore (Hive-1) service:
hive.compactor.initiator.on = true
(default)
- In Hive on Tez service:
hive.compactor.worker.threads =
<a value greater
than 0> (default and recommended value = 5)
hive.metastore.runworker.in = hs2
(default)
You must run the compactor initiator on only one HMS instance at a time.
Tables or partitions you are compacting must be full ACID or
insert-only ACID tables.
-
In Cloudera Manager, select the Hive metastore service: .
-
Search for
compact
.
-
Check that Turn on Compactor Initiator Thread
(
hive.compactor.initiator.on),
Number of Threads Used by
Compactor (hive.compactor.worker.threads
), and Run Compactor on
Hive Metastore or HiveServer2 (hive.metastore.runworker.in
) are
set to the values shown above.
-
Save the changes.
-
In Cloudera Manager, select the Hive on Tez service: .
-
Search for
compact
.
-
Check that the Number of Threads Used by Compactor
(
hive.compactor.worker.threads
), and Run compactor on Hive
Metastore or HiveServer2 (hive.metastore.runworker.in
) is set
to hs2.
-
Save the changes and restart the Hive on Tez and Hive (HIVE-1) metastore
services at an appropriate time.