Upgrading a Data Lake

If a new Runtime/CM version or build is available for the Data Lake, you can initiate an upgrade from the Management Console. An OS upgrade may also be available.

In most cases it is not required that you destroy/recreate any Data Hubs attached to the Data Lake cluster. For major/minor version upgrades, you must upgrade the Data Hubs themselves after you upgrade the Data Lake, with the exception of Data Hubs on Runtime version 7.2.16 and later. If your Data Hub is on Runtime version 7.2.16 or later, it is compatible with a Data Lake on a newer Runtime version (7.2.17+). You can independently upgrade your Data Hubs at a later time if you choose to, though it is not required.

Any Data Hubs or data services that are not stopped during a Data Lake upgrade will error out during the upgrade process.

Required role: EnvironmentAdmin or Owner over the environment

Stop all Data Hubs attached to the environment.
From the Management Console, click Data Lakes > <Environment Name>, scroll to the bottom of the Data Lake details page, and click the Upgrade tab.
Click the Target Cloudera Runtime Version drop-down menu to see any available upgrades for a given Runtime version.
If a new build is available for the selected version, the UI displays the current and target versions and build numbers. If only an OS upgrade is available, the UI displays “(OS upgrade only).”

When a major/minor version upgrade is available, you'll be able to select a new Runtime version:

If a rolling upgrade is available, select the Perform rolling upgrade checkbox if you would like to perform this type of upgrade. The availability of a rolling upgrade depends on the current and target Runtime versions, the Data Lake shape, and the Data Lake OS. See Data Lake rolling upgrades for more information.
If you want to skip the automatic backup that is taken before the upgrade, uncheck the Automatic backup box. For more information on what is backed up during a Data Lake backup, see Data Lake backup and restore.
Click Validate and Prepare to check for any configuration issues and begin the Cloudera Runtime parcel download and distribution. Using the validate and prepare option does not require downtime and makes the maintenance window for an upgrade shorter. Validate and prepare also does not make any changes to your cluster and can be run independently of the upgrade itself. Although you can begin the upgrade without first running the validate and prepare option, using it will make the process smoother and the downtime shorter.
Click Upgrade to initiate the upgrade.
Click the Event History tab to monitor the upgrade process and verify that it completes successfully.
If the upgrade fails for any reason, check the Data Lake logs through Cloudera Manager for troubleshooting information and retry the upgrade. If you cannot fix the problem manually, you may be able to recover the Data Lake cluster after a failed upgrade. For more information see Recovering from failed upgrades.

For major/minor upgrades, if the upgrade is successful, you can proceed to upgrading your attached Data Hubs if required. Data Hub clusters must run the same Runtime version as the Data Lake, with the exception of Data Hubs on Runtime version 7.2.16 and later. If your Data Hub is on Runtime version 7.2.16 or later, it is compatible with a Data Lake on a newer Runtime version (7.2.17+). You can independently upgrade your Data Hubs at a later time if you choose to, though it is not required. For service pack and OS upgrades, you can restart your Data Hubs, data services, and any stopped Virtual Warehouses.