Upgrading with the EMC DSSD D5

Minimum Required Role: Cluster Administrator (also provided by Full Administrator)

You must perform the following procedure when upgrading the DSSD D5 between minor or maintenance versions of CDH 5. See the product compatibility matrix for Product Compatibility Matrix for EMC DSSD D5.

Before you begin:
  • Review the DSSD D5 Installation guide (available from EMC) to verify that dependencies for the DSSD Hadoop Plugin are satisfied.
  • Ensure Java 1.7 or 1.8 is installed across the cluster. For installation instructions and recommendations, see Java Development Kit Installation or Upgrading to Oracle JDK 1.8, and make sure you have read Known Issues and Workarounds in Cloudera Manager 5 before you proceed with the upgrade.
  • To avoid a high number of alerts during the upgrade process, you can enable maintenance mode on your cluster before you start the upgrade. This stops email alerts and SNMP traps from being sent, but does not stop checks and configuration validations. Exit maintenance mode when you have finished the upgrade to reenable Cloudera Manager alerts.
  • When upgrading Impala, whether in CDH or a standalone parcel or package, check your SQL against the newest reserved words listed in incompatible changes. If upgrading across multiple versions or in case of any problems, check against the full list of Impala keywords.
  • If you enable high availability for HDFS, you can restart your cluster after the upgrade using a rolling restart procedure that does not require cluster down time.
To upgrade a cluster that uses the DSSD D5 storage appliance for DataNodes:
  1. Upgrade Cloudera Manager. See Upgrading Cloudera Manager 5 Using Packages.
  2. Open the Cloudera Manager Admin Console and go to Hosts > Parcels.
  3. Click Configuration.
  4. In the Remote Parcel Repository URLs section, click and add the following remote parcel repository URL: 
    http://cloudera-dssd.s3-website-us-west-1.amazonaws.com/dhp1/parcels/1.3
  5. Click Save Changes.
  6. Locate the DSSD 1.3 and DSSD_SCR 1.3 parcels and click the Download button for each parcel.
  7. After the parcels have downloaded, click the Distribute button for each parcel.
  8. If you are upgrading from CDH 5.8 to CDH 5.9 or higher, back up HDFS metadata using the following command: 
    hdfs dfsadmin -fetchImage local directory
  9. From the Home > Status tab, click next to the cluster name and select Upgrade Cluster. The Upgrade Wizard starts.
  10. If the option to choose between packages and parcels displays, select the Use Parcels option.
  11. Read the notices for steps you must complete before upgrading, click the Yes, I ... checkboxes after completing the steps, and click Continue.
  12. Cloudera Manager checks that hosts have the correct software installed. Click Continue.
  13. The selected parcels are downloaded and distributed. Click Continue.
  14. The Host Inspector runs and displays the CDH version on the hosts. Click Continue.
  15. Choose the type of upgrade and restart:
    • Rolling Restart (Default) - Cloudera Manager upgrades services and performs a rolling restart. This option is available only if you have enabled high availability for HDFS.
      1. Choose the batch size (default 1) described in Performing a Cluster-Level Rolling Restart and click Advanced Options to specify additional rolling restart options. Services that do not support rolling restart undergo a normal restart and are not available during the restart process.
      2. Click Continue. The Upgrade Cluster Command screen displays the result of the commands run by the wizard as it shuts down services, activates the new parcel, upgrades services, deploys client configuration files, restarts services, and performs a rolling restart of the services that support it.
    • Full Cluster Restart - Cloudera Manager performs all service upgrades and restarts the cluster, during which services are not available.
      1. Click Continue. The Upgrade Cluster Command screen displays the result of the commands run by the wizard as it shuts down all services, activates the new parcel, upgrades services, deploys client configuration files, and restarts services.
      2. Click Continue. The wizard reports the result of the upgrade.
    • Manual Upgrade - Cloudera Manager configures the cluster to the specified CDH version but performs no upgrades or service restarts. Upgarding manually is complex and for advanced users only. Cloudera Manager displays links to documentation describing the required upgrade steps. Click Continue and then follow the DSSD-specific steps at this link instead: Manual CDH Upgrade Steps for DSSD D5.
  16. Click Finish to return to the Home page.
  17. If you are upgrading from CDH 5.8 to CDH 5.9 or higher:
    Finalize the HDFS metadata upgrade. To determine when to finalize the upgrade, run important workloads and ensure that they are successful. After you have finalized the upgrade, you cannot roll back to a previous version of HDFS without using backups. Verifying that you are ready to finalize the upgrade can take a long time. Make sure you have enough free disk space, keeping in mind that the following behavior continues until the upgrade is finalized:
    • Deleting files does not free up disk space.
    • Using the balancer causes all moved replicas to be duplicated.
    • All on-disk data representing the NameNodes metadata is retained, which could more than double the amount of space required on the NameNode and JournalNode disks.
    To finalize the metadata upgrade:
    If you have enabled high availability for HDFS:
    1. Go to the HDFS service.
    2. Select Actions > Finalize Rolling Upgrade and click Finalize Rolling Upgrade to confirm.
    If you have not enabled high availability for HDFS:
    1. Go to the HDFS Service.
    2. Select the Instances tab.
    3. From the list of Roles, click NameNode.
    4. Select Actions > Finalize Metadata Upgrade .