Configuring the Hive Metastore to Use HDFS High Availability in CDH

In this task, you configure a host for connecting to the backend database. You can either configure environment variables or add configuration properties to hive-site.xml. Configure a limited number of hosts to limit exposing the backend Hive database username, password, and connection string. To configure environment variables and run the metatool, follow this procedure:

  1. Connect to any Hive Gateway host, Hive Metastore (HMS), or HiveServer (HS2) host.
  2. Set HIVE_CONF_DIR to the Hive MetaStore process directory.
  3. Set HADOOP_CREDSTORE_PASSWORD to the same value defined in the supervisor.conf file (which is located in the Hive MetaStore process directory)
  4. Set AUX_CLASSPATH to the value of your your Hive MetaStore process directory, which is defined in logs/stderr.log.
    For example: 
    export HIVE_CONF_DIR=/var/run/cloudera-scm-agent/process/4595-hive-HIVEMETASTORE
export HADOOP_CREDSTORE_PASSWORD=abcdefg1234...
export AUX_CLASSPATH=/opt/cloudera/parcels/CDH-5.13.3-1.cdh5.13.3.p0.2....
  5. Run the following command to connect to the database and list FS roots: 
    hive --service metatool -listFSRoot
    The output is: 
    Listing FS Roots..
hdfs://[hostname]:8020/user/hive/warehouse

Alternatively, instead of setting and exporting environment variables, open the hive-site.xml file in /etc/hive/conf/. Add the following properties to the hive-site.xml:

<property>
   <name>javax.jdo.option.ConnectionURL</name> 
   <value>jdbc:mysql://[ENTER BACKEND DATABASE HOSTNAME]:[ENTER PORT]/[ENTER HIVE BACKEND DATABASE USERNAME]?useUnicode=true&characterEncoding=UTF-8</value>
</property>
<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>[ENTER BACKEND DATABASE USERNAME]</value>
</property>
<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>[ENTER BACKEND DATABASE PASSWORD]</value>
</property>

To determine what the backend database host, port, username, or password is, in Cloudera Manager, go to Hive > Configurations. Set Category Filter to Hive Metastore Database. The password is not exposed in plaintext.

To configure other CDH components to use HDFS high availability, see Configuring Other CDH Components to Use HDFS HA.