Rolling Upgrade Prerequisites
To perform a Rolling Upgrade, your cluster must meet the following prerequisites. If your cluster does not meet these upgrade prerequisites, you can consider an Express Upgrade.
HDFS
- NameNode HA
NameNode HA must be enabled and working properly. From Ambari Web, the NameNodes, ZKFailoverControllers and JournalNodes must be “green” and showing no alerts.
- NameNode Truncate
In HDP 2.2, truncate must be disabled and this prerequisite is REQUIRED. The hdfs-site configuration dfs.allow.truncate, property is set to true if truncate is enabled. If the property is not set or is false, truncate is disabled.
In HDP 2.3 and later, truncate is always enabled, which is acceptable. Therefore, this check is not applicable and will not be performed with HDP 2.3 maintenance upgrades.
YARN
- Timeline Service State Recovery
YARN state recovery should be enabled. Check the Services > YARN > Configs > Advanced property yarn.timeline-service.recovery.enabled is set to true.
- ResourceManager Work Preserving Recovery
YARN work preserving recovery should be enabled. Check the Services > YARN > Configs > Advanced property yarn.resourcemanager.work-preserving-recovery.enabled is set to true.
- ResourceManager HA
ResourceManager HA should be enabled to prevent a disruption in service during the upgrade.
This prerequisite is OPTIONAL.
MapReduce2
- MapReduce Distributed Cache
MapReduce should reference Hadoop libraries from the distributed cache in HDFS.
- JobHistory State Recovery
JobHistory state recovery should be enabled. Check the following: Services > MapReduce > Configs > Advanced property mapreduce.jobhistory.recovery.enable is set to true . Once enabled, dependent properties for mapreduce.jobhistory.recovery.store.class and mapreduce.jobhistory.recovery.store.leveldb.path should also be set.
- Encrypted Shuffle
If encrypted shuffle has been enabled, the
ssl-client.xml
file must be copied to/etc/hadoop/conf/secure
directory on each node in the cluster.
Tez
- Tez Distributed Cache
Tez should reference Hadoop libraries from the distributed cache in HDFS. Check the Services > Tez > Configs configuration and confirm tez.lib.uris is set with the path in HDFS for the
tez.tar.gz
file.
Hive
- Hive Dynamic Service Discovery
HiveServer2 dynamic service discovery should be configured for Rolling Upgrade.
- Hive Client Retry
Hive client retry properties must be configured. Review the Services > Hive > Configs configuration and confirm hive.metastore.failure.retries and hive.metastore.client.connect.retry.delay are specified.
- Multiple Hive Metastore
Multiple Hive Metastore instances are recommended for Rolling Upgrade. This ensures that there is at least one Hive Metastore running during the upgrade process. Additional Hive Metastore instances can be added through Ambari.
This prerequisite is OPTIONAL.
Oozie
- Oozie Client Retry
Oozie client retry properties must be configured. Review the Services > Oozie > Configs > oozie-env configuration and confirm "export OOZIE_CLIENT_OPTS="${OOZIE_CLIENT_OPTS} -Doozie.connection.retry.count=<number of retries>" is specified.
Druid
- Druid High Availability
Enable Druid high availabilty (HA) before performing a rolling upgrade. Multiple instances of Druid services running in the same cluster avoid downtime during the upgrade. If you choose not to enable HA for Druid services, a warning message appears but you can start the upgrade with the risk that Druid might encounter downtime.
More Information
Configuring NameNode High Availability.