Data Lake upgrade

When important changes are available for the Data Lake service, you can initiate a maintenance (often called a "hotfix") upgrade and/or an OS upgrade. Use either the CDP CLI or the Management Console to initiate an upgrade.

The maintenance/hotfix upgrade process checks to see if a new Cloudera Manager or Cloudera Runtime (CDP) build is available, and then upgrades the Data Lake to the newest builds. Maintenance upgrades do not upgrade to a new version of Cloudera Manager or Cloudera Runtime; they only upgrade to the latest build of a particular version. For example, you cannot upgrade from Runtime version 7.2.8 to version 7.2.9, but you can upgrade Runtime 7.2.9 to a newer 7.2.9 build. These upgrades are made available as needed and can deliver targeted bug fixes for Cloudera Runtime, Cloudera Manager, or both. If desired, specific older hotfixes can also be chosen using the CDP CLI.

OS upgrades may be available sporadically as new images are created. OS upgrades are typically released to address security vulnerabilities or other issues on the host system. The OS upgrade process includes launching entirely new instances with the new OS image.

The type of upgrade that is available for a Data Lake is explicit on the Upgrade tab of the Data Lake details page. Any available upgrades are visible in the Target Cloudera Runtime Version drop-down menu.

Process

The Data Lake upgrade process will:
  1. Check for a newer Cloudera Manager and Runtime build, and a new OS image.
  2. Execute the Data Lake upgrade.
  3. Verify the Data Lake state.

Before You Begin

  • The Data Lake must be running and in a healthy state.
  • Experiences should run the latest releases that are available to the service.
  • Plan the upgrade during a time of low activity. While it is not required to stop any Experiences or destroy and re-create any Data Hubs, any Data Hubs or Experiences that are not stopped will error out during the upgrade process. For the Cloudera Data Warehouse Experience, you should stop any Virtual Warehouses that are running prior to beginning any upgrade or backup/restore process.
  • Plan for at least two hours of downtime while the upgrade completes.
  • The upgrade requires 27 GB space on the CM server node and 20 GB on every other instance. If space is insufficient on your Data Lake, upgrade will not be permitted.

Limitations

  • Data Lake upgrade does not include the upgrade of the FreeIPA software or the operating system on the instance(s) running FreeIPA. No upgrades are supported for FreeIPA at this time.
  • Data Lake resizing during an upgrade is not supported at this time.
  • Rolling upgrade of SDX services is not supported at this time.