6. Upgrade Apache Hive and Apache HCatalog

In HDP 2.1 (Apache Hive 0.13.0) the Decimal data type is now treated as the type Decimal(10,0): 10 digits of precision and 0 scale. This is a change from the variable precision and scale that was available in Hive 0.11.0 and Hive 0.12.0, which allowed up to 38 digits of precision and unlimited scale.

In order to avoid unintended "rounding" of decimal data, sites that were previously running Hive 0.11.0 and Hive 0.12.0 may need to migrate tables with Decimal columns after upgrading to Hive 0.13.0. For details, see the Apache Hive wiki. For assistance with upgrades that involve Decimal data, please contact Hortonworks Support.

Stop the Hive Metastore service, if you have not done so already.
Note
Make sure that the Hive metastore database is running.
Upgrade Hive and HCatalog. On the Hive and HCatalog host machines, execute the following commands:
- For RHEL/CentOS:
```
yum upgrade hive
yum erase hcatalog
yum install hive-hcatalog 
```
- For SLES:
  Run the following commands:
```
zypper rm hive hcatalog
zypper install hive-hcatalog
zypper install bigtop-tomcat
```
- For Ubuntu or Debian:
```
apt-get update hive hcatalog
```
Upgrade the Hive Metastore database schema.
```
$HIVE_HOME/bin/schematool -upgradeSchema -dbType <$databaseType> 
```
The value for $databaseType can be derby, mysql, oracle, or postgres.
Download and extract the HDP companion files. Copy the hive-site.xml file in the configuration_files/hive directory of the extracted companion files to the etc/hive/conf directory on your Hive host machine. This new version of the hive-site.xml file contains new properties for HDP-2.1 features.

Edit the hive-site.xml file and modify the properties based on your environment. Search for TODO in the file for the properties to replace.

Edit the connection properities for your Hive metastore database in hive-site.xml:

<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://TODO-HIVE-METASTORE-DB-SERVER:TODO-HIVE-METASTORE-DB-PORT/TODO-HIVE-METASTORE-DB-NAME?createDatabaseIfNotExist=true</value>
    <description>Enter your Hive Metastore Connection URL, for example if MySQL: jdbc:mysql://localhost:3306/mysql?createDatabaseIfNotExist=true</description>    
  </property>
    
  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>TODO-HIVE-METASTORE-DB-USER-NAME</value>
    <description>Enter your Hive Metastore database user name.</description>
  </property>
  
  <property>       
   <name>javax.jdo.option.ConnectionPassword</name>       
   <value>TODO-HIVE-METASTORE-DB-PASSWORD</value>  
   <description>Enter your Hive Metastore database password.</description>
  </property>
  
  <property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>TODO-HIVE-METASTORE-DB-CONNECTION-DRIVER-NAME</value>
  <description>Enter your Hive Metastore Connection Driver Name, for example if MySQL: com.mysql.jdbc.Driver</description>
</property>

Optional: If you want Hive Authorization, set the following Hive authorization parameters in the hive-site.xml file:

<property>
  <name>hive.security.authorization.enabled</name>
  <value>true</value>
</property>
  
<property>
  <name>hive.security.authorization.manager</name>
  <value>org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider</value>
</property>

<property>
  <name>hive.security.metastore.authorization.manager</name>
  <value>org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider</value>
</property>

<property>
  <name>hive.security.authenticator.manager</name>
  <value>org.apache.hadoop.hive.ql.security.ProxyUserAuthenticator</value>
</property>

For a remote Hive metastore database, use the following hive-site.xml property value to set the IP address (or fully-qualified domain name) and port of the metastore host. To enable HiveServer2, leave this property value empty.

<property>       
 <name>hive.metastore.uris</name>       
 <value>thrift://$metastore.server.full.hostname:9083</value>  
 <description>URI for client to contact metastore server. To enable HiveServer2, leave the property value empty. </description>
</property>

	Note
	You can also use the HDP utility script to fine-tune your configuration settings based on node hardware specifications.

Disable autocreation of schemas to match HDP 2.1 configurations.

Edit hive-site.xml to set the value of datanucleus.autoCreateSchema to false.

<property>
  <name>datanucleus.autoCreateSchema</name>
  <value>false</value>
  <description>Creates necessary schema on a startup if one doesn't exist.</description>
</property>

Start Hive. On the Hive Metastore host machine, execute the following command:

sudo su -l $HIVE_USER -c "nohup hive --service metastore > $HIVE_LOG_DIR/hive.out 2> $HIVE_LOG_DIR/hive.log &"

Start Hive Server2. On the Hive Server2 host machine, execute the following command:
```
sudo su -l $HIVE_USER -c "nohup /usr/lib/hive/bin/hiveserver2 -hiveconf hive.metastore.uris=\" \" > $HIVE_LOG_DIR/hiveserver2.out 2> $HIVE_LOG_DIR/hiveserver2.log &"                 
```
where
- $HIVE_USER is the Hive Service user. For example, hive.
- $HIVE_LOG_DIR is the directory where Hive server logs are stored (example: /var/log/hive).

Legal notices

	Note
	Make sure that the Hive metastore database is running.