Upgrade process

Read through the complete information about the upgrade options, prerequisites, and the overall process before starting the upgrade. Cloudera recommends that you validate these steps in a test environment to adjust and account for any special configurations for your cluster.

The high-level process for performing an HDP intermediate bits upgrade is as follows:

Ambari will guide you through the steps required to upgrade the HDP intermediate bits. Make sure Ambari and the cluster are healthy, operating normally, and all service checks are passing. Ambari 2.7.5.x supports express methods for upgrading HDP 3.1.5.x to HDP 7.1.x.x. An Express Upgrade orchestrates the HDP upgrade in an order that will incur cluster downtime.

note

In the HDP 3.1.5.x cluster, Hive hook and HBase hook are enabled by default.
SAC (Spark Atlas Connector or Spark Hook) is disabled by default.
Note that, in HDP, SAC is in a Technical Preview state. In CDP Private Cloud Base, Hive Hook and HBase hook are enabled by default, In the AM2CM upgrade flow, if the Hive Hook and HBase hook are enabled in the HDP 3.1.5.x cluster, post-upgrade, it is enabled in CDP Private Cloud Base.
HMS Hook functionality did not exist in HDP 3.1.5.x, hence it must be manually enabled in CDP Private Cloud Base.
Similarly SAC in CDP Private Cloud Base must also be manually enabled.

note

Intermittently, the default owner permissions for /warehouse/tablespace/external/hive directory is updated to activity_analyzer user from the smartsense service in the HDP3 cluster, because of these permissions issues the hive external table creation fails. The workaround is to Kinit as the HDFS user and update the permissions for warehouse directory using HDFS cli.

$ hdfs dfs -chown hdfs:hdfs /warehouse
$ hdfs dfs -chown hdfs:hdfs /warehouse/tablespace
$ hdfs dfs -chown hdfs:hdfs /warehouse/tablespace/external
$ hdfs dfs -chown hdfs:hdfs /warehouse/tablespace/external/hive