Apache Tez processing of Hive jobs
If you were running Hive on HDP or CDP, you have been running Hive queries using the Apache Tez execution engine. Hive in CDW PvC also uses Tez to run queries and is a HiveServer2 endpoint as it is in HDP or CDP. Learn how Tez processes Hive jobs in CDP and CDW and understand the tasks that you need to perform after migrating your workloads to CDW.
Hive is fundamentally the same technology in HDP, CDP Private Cloud Base, and CDW Private Cloud. Hive syntax and semantics are basically the same after upgrading from HDP to CDP Private Cloud or to CDW Private Cloud.
- Container mode — Every time you run a Hive query, Tez requests a container from YARN.
- LLAP mode — Every time you run a Hive query, Tez asks the LLAP daemon for a free thread, and starts running a fragment.
In CDW, the Hive execution mode is LLAP. In Cloudera Data Hub on CDP Public Cloud and CDP Private Cloud Base, the Hive execution mode is container, and LLAP mode is not supported. When Apache Tez runs Hive in container mode, it has traditionally been called Hive on Tez.
There are certain differences between Hive on Tez and LLAP that you need to be aware of before migrating to CDW Private Cloud.
- The HiveServer2 endpoints authenticate using LDAP instead of Kerberos.
- Your old Hive JDBC drivers need to be replaced with the latest drivers.
- If you have Hive User-Defined Functions (UDFs) in CDP Private Cloud Base then the UDF JARs have to be added to the CDW Hive classpath and registered.
After migrating to CDW Private Cloud, perform the following tasks:
- Download the latest Hive JDBC drivers from the Hive JDBC driver download page and follow the driver installation instructions on the download page.
- Update the JDBC client connection URL to point to the Virtual Warehouse instance of HiveServer2.
- If your previous connection in CDP PvC Base used Kerberos for authentication, you must modify the connection URL accordingly.
- Ensure that the UDF JARs are added to the