Apache Hive Performance Tuning
Also available as:
PDF

Hive Setup for Using Multiple Queues

For multiple workloads or applications, using multiple HiveServer2 instances is recommended. Each HiveServer2 instance can have its own settings for Hive and Tez.

Figure 3.5. Hive Setup Using Multiple Queues


Installing a Second Instance of HiveServer2

The following figure shows multiple HiveServer2 instances.

Figure 3.6. Multiple HiveServer2 Installations


To install a second instance of HiveServer2 manually:

[Note]Note

The following is a summarized task list for installation. For more information, see the Installing Apache Hive and Apache HCatalog chapter of the Non-Ambari Cluster Installation Guide. It is recommended that you validate the installations as documented in the guide.

  1. yum install hive hcatalog hadoop hadoop-hdfs hadoop-libhdfs
            hadoop-yarn hadoop-mapreduce hadoop-client openssl

    If the new node is part of the cluster, which means it already has Hadoop and HDFS installed on it, then you only need to install Hive and HCatalog:

    yum install hive hcatalog
    [Note]Note

    Installing HCatalog separately is required for HDP 1.3.x because Hive and HCatalog were not merged in that release.

  2. Copy the following configuration files from the original HiveServer2 instance to the new HiveServer2 instance:

    • hive-site.xml and hiveserver2-site.xml (in HDP 2.2 and later) located under /etc/hive/conf

    • core-site.xml, hdfs-site.xml, mapred-site.xml, yarn-site.xml located under /etc/hadoop/conf

  3. Copy the database driver for the backing metastore DB (for example postgresql-jdbc.jar for Postgre) from the lib folder of the original HiveServer2 instance to the lib folder of the new HiveServer2 instance.

  4. Start the HiveServer2 service:

    su $HIVE_USER
    /usr/lib/hive/bin/hiveserver2 -hiveconf hive.metastore.uris=" "
    -hiveconf hive.log.file=hiveserver2.log
    >$HIVE_LOG_DIR/hiveserver2.out 2
    >$HIVE_LOG_DIR/hiveserver2.log &
  5. Connect to the new HiveServer2 instance by using Beeline and validate that it is running:

    1. Open the Beeline command line shell to interact with HiveServer2:

      /usr/bin/beeline
    2. Establish a connection to the server:

      !connect jdbc:hive2://$hive.server.full.hostname:10000
      $HIVE_USER password org.apache.hive.jdbc.HiveDriver
    3. Run sample commands:

      show databases;
      create table test2(a int, b string);
      show tables;

To install a second instance of HiveServer2 with Ambari:

  1. If the new HiveServer2 instance is on a new host that has not yet been added to the cluster:

    1. Open Ambari and navigate to the Hosts tab.

    2. Follow the wizard instructions to add the new host to the cluster.

  2. Navigate to the Hive services page.

  3. Under Service Actions, select Add HiveServer2, and follow the wizard instructions to add the HiveServer2 instance.

For information about adding hosts to a cluster or adding services with Ambari, see the Ambari User's Guide.